-- Logs begin at Tue 2022-10-11 12:47:33 UTC, end at Tue 2022-10-11 13:52:53 UTC. -- Oct 11 13:13:30.289707 dv-cinder-850618-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 11 13:13:30.297162 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 11 13:13:30 2022] *** Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: nodename: dv-cinder-850618-5-iscsi Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: machine: x86_64 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: clock source: unix Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: pcre jit disabled Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: detected number of CPU cores: 4 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: current working directory: / Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: detected binary path: /usr/bin/uwsgi-core Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: your processes number limit is 47680 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: your memory page size is 4096 bytes Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: detected max file descriptor number: 2048 Oct 11 13:13:30.299160 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: lock engine: pthread robust mutexes Oct 11 13:13:30.300067 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: thunder lock: enabled Oct 11 13:13:30.300067 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 11 13:13:30.300067 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 11 13:13:30.338934 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: Python main interpreter initialized at 0x5637d15629c0 Oct 11 13:13:30.338934 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: python threads support enabled Oct 11 13:13:30.338934 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: your server socket listen backlog is limited to 100 connections Oct 11 13:13:30.338934 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: your mercy for graceful operations on workers is 90 seconds Oct 11 13:13:30.339342 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: mapped 403077 bytes (393 KB) for 2 cores Oct 11 13:13:30.339408 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: *** Operational MODE: preforking *** Oct 11 13:13:30.339682 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: *** uWSGI is running in multiple interpreter mode *** Oct 11 13:13:30.339682 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: spawned uWSGI master process (pid: 81357) Oct 11 13:13:30.339895 dv-cinder-850618-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 11 13:13:30.340986 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: spawned uWSGI worker 1 (pid: 81370, cores: 1) Oct 11 13:13:30.340986 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: spawned uWSGI worker 2 (pid: 81371, cores: 1) Oct 11 13:13:30.340986 dv-cinder-850618-5-iscsi devstack@keystone.service[81357]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 11 13:13:34.527533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 13:13:34.527533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Oct 11 13:13:34.528385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 13:13:34.528385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: from cryptography.utils import int_from_bytes Oct 11 13:13:34.528797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 13:13:34.528797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Oct 11 13:13:34.528905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 13:13:34.528905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: from cryptography.utils import int_from_bytes Oct 11 13:13:34.577250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 13:13:34.577431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 13:13:34.577533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 13:13:34.577613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 13:13:34.577678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] command line args: [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 13:13:34.577737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] command line args: [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 13:13:34.577868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 13:13:34.578018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 13:13:34.578105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ================================================================================ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 13:13:34.578180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ================================================================================ {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 13:13:34.578251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] admin_token = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] config_dir = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] config_dir = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] config_source = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] config_source = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.578967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] debug = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] debug = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.579921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_config_append = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_config_append = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.580954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_dir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_dir = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_file = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.581887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.582869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.583950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.584149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.584319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.584436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.584526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.584923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] publish_errors = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] publish_errors = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.585945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] standard_threads = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] standard_threads = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.586997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] use_journal = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] use_journal = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] use_json = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] use_json = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] use_stderr = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] use_stderr = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] use_syslog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] use_syslog = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.587915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 13:13:34.588080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.588852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.589977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.590968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.591990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.592974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.593992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.594899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.595981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.596934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.597964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.598949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.599958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.600869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.601969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.602986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.603936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.604930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.605985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.606926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.607922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.608897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609052 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.609973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.610957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.611886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.612910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.external = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.external = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.613989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.password = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.password = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.token = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] auth.token = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.614996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.615934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.616897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.617851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.618863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.619988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.620949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.621974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.622933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.623878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.624867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.625916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.626958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.627910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.628856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.629943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.630953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.631870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.632923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.633950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.634882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.635984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.636947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.637946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.638896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.639986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.640953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.641980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.642913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.643993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.644981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.645982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] role.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] role.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] role.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] role.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.646988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.647952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.648966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649671 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.649908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.650923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.651966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.652936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.653887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.654963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655671 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.655921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.656978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.657916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.658893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.connection = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.659868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.660981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.661996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.662933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 13:13:34.663697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 13:13:34.663769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 13:13:34.692846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.692991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.693058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.693119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.693283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.693367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.701660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.701753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.729186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.729326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.729481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.729608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.781539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.781691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.829183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.829323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.829393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.829479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.986573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.986866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.987039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.987234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.987386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.987608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.987886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.988083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.988351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.988573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.988758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:34.988991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.010940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.011739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.022954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.023218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.035748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.035880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.035950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.036971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81370) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.037034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.037219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 13:13:35.053642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.053888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 13:13:35.054185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.054307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.054376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 13:13:35.054459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 13:13:35.054646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.054800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.054956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 13:13:35.055196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.055981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.065958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.066066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.066371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.072925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.073408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.075808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.076090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.076350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.076477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.076548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.076656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.076830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.076954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.077902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.078982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.101086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.101505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.102622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.102897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.103020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.103268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.107565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.107850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.107975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.108106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.108240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.108455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.114502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.114740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.114866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.115018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.120436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.120644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.120743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.120904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.120996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.121855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.122038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.122163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.122309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.122433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.122545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.137875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.138000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.138159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.138284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.142033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.142178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.145268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.145393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.145566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.145691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.150124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.150250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.153250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.153374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.156472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.156616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.156769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.156894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.157896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.158041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.173762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.173886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.174047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.174170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.174316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.174459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.174532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.174646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.181430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.181555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.182970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.183149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.184526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.184651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.184803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.184928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.187489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.187627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.187746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.187839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.188917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.199908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.200027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.200244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.200372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.200441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.200541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.200754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.200885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.201813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.216954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.217099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.219835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.219961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.222731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.222857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.224146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.224389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.225672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.225877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.225976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.226145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.226290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.226584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.232747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.232993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.234181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.234373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.237070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.237306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.237433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.237701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.237914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.238012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.238078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.238250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.245753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.246040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.246183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.246301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.246394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.246611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.253185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.253465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.253587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81370) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 13:13:35.253741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.253881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.253985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.254822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.255954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.256932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81370) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 13:13:35.290779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5637d15629c0 pid: 81371 (default app) Oct 11 13:13:35.292664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5637d15629c0 pid: 81370 (default app) Oct 11 13:13:35.294010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f514-f542-4607-962f-6dfb652f2d8a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:35.294291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f514-f542-4607-962f-6dfb652f2d8a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:35.294561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f514-f542-4607-962f-6dfb652f2d8a None None] PATH_INFO: `/v3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:35.304488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Tue Oct 11 13:13:35 2022] GET /identity/v3/ => generated 255 bytes in 13 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:42.094412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:42.095256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:42.095256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:42.102235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:42.109251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:42.109711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:42.109848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:42.189110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: INFO dbcounter [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] Registered counter for database keystone Oct 11 13:13:42.210071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG oslo_db.sqlalchemy.engines [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 13:13:42.218027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writer thread running {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 13:13:42.412483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 13:13:42.412948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 13:13:42.478232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:42.479834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:42.613363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:13:43.053881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:43.125970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: INFO dbcounter [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] Registered counter for database keystone Oct 11 13:13:43.150450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG oslo_db.sqlalchemy.engines [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 13:13:43.161590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writer thread running {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 13:13:43.285897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1d23c-ced5-4052-87f7-61fe7f44234c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:43.291373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:43.291999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:43.292303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:43.292570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:43.301779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:43.357996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 13:13:43.358260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e29595-96aa-45ca-a6af-e9b4c846c03f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:43.368595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 11 13:13:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:13:43.375777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:43.398239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11e7ab-8d82-4bc3-976f-bcbce33e69fe None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:43.404781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:43.405908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:43.406194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:43.406467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] PATH_INFO: `/v3/projects/81928fdc604942f18e8f5f0944263250` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:43.408373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] RBAC: Authorizing `identity:get_project(project_id=81928fdc604942f18e8f5f0944263250)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:43.460081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 13:13:43.460467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc11e23-0dd3-4fb8-8473-d0d64387e582 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:43.465071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:13:43 2022] GET /identity/v3/projects/81928fdc604942f18e8f5f0944263250 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:45.557243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b79f-a9ad-4c3d-baca-215ebcbee775 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:45.558068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b79f-a9ad-4c3d-baca-215ebcbee775 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:45.558068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b79f-a9ad-4c3d-baca-215ebcbee775 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:45.564990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:45.571138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:45.571563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:45.571715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:45.592453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:45.593967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:45.601766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:13:45.865272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639b79f-a9ad-4c3d-baca-215ebcbee775 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:45.882929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639b79f-a9ad-4c3d-baca-215ebcbee775 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:45.889900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:45.890808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:45.891209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:45.891624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:45.905750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:45.909034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fed76-28ce-4274-bc18-a55673b363a6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:45.920239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 11 13:13:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:13:45.925727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:45.935576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242cc6b-2d0d-45e9-a00b-f6a6f249786f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:45.941208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:45.941840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:45.942107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:45.942377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] PATH_INFO: `/v3/users/c24104e3c5bf48be865c100865a4abd3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:45.943918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] RBAC: Authorizing `identity:get_user(user_id=c24104e3c5bf48be865c100865a4abd3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:45.945881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee914e-70e6-4018-82f2-8524dec2ed6d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:45.947498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Tue Oct 11 13:13:45 2022] GET /identity/v3/users/c24104e3c5bf48be865c100865a4abd3 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:48.093991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2c91-4926-4748-a833-4424a9616cdd None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.094558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2c91-4926-4748-a833-4424a9616cdd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.094558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea2c91-4926-4748-a833-4424a9616cdd None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.095864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:13:48.101163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.101647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.101780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.120654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:48.121826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:48.171019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655802-b0a8-4767-9082-edabc8365a48 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.171251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655802-b0a8-4767-9082-edabc8365a48 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.171560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655802-b0a8-4767-9082-edabc8365a48 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.172919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:48.178920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.179204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.179538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.208516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:48.224355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG passlib.handlers.bcrypt [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81370) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 13:13:48.224752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG passlib.handlers.bcrypt [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81370) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 13:13:48.273241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:48.274839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:48.373957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:48.471643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.495938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:48.579662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f521a-75a3-4ddf-b7ce-9f8ad9ed723b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:48.613271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedea65-b346-45db-9401-633c59036624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:48.613913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedea65-b346-45db-9401-633c59036624 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.614189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedea65-b346-45db-9401-633c59036624 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.614461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedea65-b346-45db-9401-633c59036624 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.625747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedea65-b346-45db-9401-633c59036624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ffedea65-b346-45db-9401-633c59036624 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:13:48.634637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:13:48.635572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:13:48.636436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:13:48.637136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.641698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedea65-b346-45db-9401-633c59036624 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.722218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:48.739198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255fbee-3436-4d9c-bb23-39179a54a2ea None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:48.741178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedea65-b346-45db-9401-633c59036624 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:48.741599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:48.776272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:48.776476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:48.777030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.777136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.777308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.777405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.777577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.777668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.778243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:13:48.778544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:48.778626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:48.779937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:48.780027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:48.791855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:13:48.792074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.sql.core [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:13:48.792074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:13:48.792074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [parameters: {'id': '63c244d0f1974a518a589417d7ae5395', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 13:13:48.792074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81370) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 13:13:48.797462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:13:48.810411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 13:13:48.811418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application yield resource Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise self.value Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application session.commit() Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 13:13:48.812697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.flush() Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application compat.raise_( Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 13:13:48.813773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 13:13:48.814856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:13:48.815759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [parameters: {'id': '63c244d0f1974a518a589417d7ae5395', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 13:13:48.816930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:13:48.818151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 8/16] 10.222.0.15 () {60 vars in 1137 bytes} [Tue Oct 11 13:13:48 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:13:48.819240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.907088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a716df-84d8-4357-af99-39847b7c084b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:48.916287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839772a5-dd13-489a-9f00-7217e840211c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:48.941133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:48.941781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.942062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.942333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] PATH_INFO: `/v3/users/c24104e3c5bf48be865c100865a4abd3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.943456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] RBAC: Authorizing `identity:get_user(user_id=c24104e3c5bf48be865c100865a4abd3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:48.944964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:48.946629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/users/c24104e3c5bf48be865c100865a4abd3 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:13:48.949153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:48.950020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:48.950292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:48.950557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:48.952071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:48.966033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:48.970423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:48.971954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:13:48.988858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:13:48.989903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:13:48.990737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:13:48.990737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:13:48.990737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:48.990737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:13:48.995934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:49.017238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:49.051260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7937-dfbd-4ab4-8db2-8a22cca96858 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:49.086044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:49.086678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:49.086952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:49.087246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:49.088389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:49.089849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:49.091383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:49.095144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91420-eb61-4c2f-bec7-a563ed42f843 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:49.097038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:49.130907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:49.131514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:49.131788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:49.132053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:49.132772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:13:49.133053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:49.134428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cecdaa-63c4-4123-b5b1-95465e005517 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:49.151698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:49.182166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:49.202148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587be54-b2aa-4c6f-831e-8e9678c857c4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:49.237997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:49.238679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:49.238951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:49.239218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] PATH_INFO: `/v3/domains/default/users/c24104e3c5bf48be865c100865a4abd3/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:49.241208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c24104e3c5bf48be865c100865a4abd3, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:49.242757 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87938121-5409-456e-8c95-2fcb5c217f3b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:49.357070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 11/21] 10.222.0.15 () {58 vars in 1361 bytes} [Tue Oct 11 13:13:49 2022] PUT /identity/v3/domains/default/users/c24104e3c5bf48be865c100865a4abd3/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:13:51.953118 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef1f1-dff8-4451-80aa-7de8ad6cacb6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:51.953651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef1f1-dff8-4451-80aa-7de8ad6cacb6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:51.953651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef1f1-dff8-4451-80aa-7de8ad6cacb6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:51.955282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:51.963697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f9e4-7c94-4e81-99ee-29a27cdfe8b6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:51.964631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f9e4-7c94-4e81-99ee-29a27cdfe8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:51.965128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f9e4-7c94-4e81-99ee-29a27cdfe8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:51.992675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d337f9e4-7c94-4e81-99ee-29a27cdfe8b6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:51.993552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3b23-abd7-4911-be51-68f9130a608f None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:51.994045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3b23-abd7-4911-be51-68f9130a608f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:51.994220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d337f9e4-7c94-4e81-99ee-29a27cdfe8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:51.994675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3b23-abd7-4911-be51-68f9130a608f None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:51.996446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:52.008860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb117df4-724e-4d7a-8e95-effff277c221 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.009425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb117df4-724e-4d7a-8e95-effff277c221 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.009886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb117df4-724e-4d7a-8e95-effff277c221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.037702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bb117df4-724e-4d7a-8e95-effff277c221 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:52.038973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bb117df4-724e-4d7a-8e95-effff277c221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:52.106949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:13:52.108045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bfa9-8234-4e01-8c8c-67c03ef5898d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.108407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bfa9-8234-4e01-8c8c-67c03ef5898d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.108627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6bfa9-8234-4e01-8c8c-67c03ef5898d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.116507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 13/25] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:52.122899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c28f-1ef2-4934-b771-3b9e0de8eb59 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.123187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c28f-1ef2-4934-b771-3b9e0de8eb59 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.123467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c28f-1ef2-4934-b771-3b9e0de8eb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.177853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7666c28f-1ef2-4934-b771-3b9e0de8eb59 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:52.179089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7666c28f-1ef2-4934-b771-3b9e0de8eb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:52.201036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 13/26] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:52.269054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:52.274671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 14/27] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:52.324686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a33b-55ce-4efc-b4d7-95c9a98412b5 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.324986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a33b-55ce-4efc-b4d7-95c9a98412b5 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.325236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a33b-55ce-4efc-b4d7-95c9a98412b5 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.326717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 14/28] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:52.341555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.341837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.342110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.369110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:52.370310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:52.458562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 15/29] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:52.549616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6a33b-55ce-4efc-b4d7-95c9a98412b5 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:52.562268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:52.665368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6a33b-55ce-4efc-b4d7-95c9a98412b5 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:52.671573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:52.737254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:52.738029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.738317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.738591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.767526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:52.769203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:13:52.795182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:13:52.796154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:13:52.796967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:13:52.796967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:13:52.796967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:13:52.798798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 15/30] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:13:52.799829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:52.825950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668875d-15ba-46f1-8d94-e7ccb8f61651 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:52.862542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:52.863400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:52.863549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:52.863827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:52.864578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:52.866164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:52.871257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:52.888549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 16/31] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 11 13:13:52 2022] POST /identity/v3/roles => generated 225 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:13:52.889767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:52.938501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.006736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ffd3d-5d60-468f-9d32-7bf9a1d644eb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.044601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11311e30-9b90-4dbf-a925-958f2880007a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.057323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.057897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.058279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.058592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.060582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.062125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eea70e-4bda-45a7-8cf3-2f24cd84f243 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.087719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.088437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.088820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.090235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.090235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:13:53.090235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.091560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.112450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 17/32] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:53.113662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.169275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.194011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 16/33] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 11 13:13:52 2022] POST /identity/v3/roles => generated 219 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:13:53.195571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f9a1-29e4-45c9-8448-1d6d75f730f8 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.195859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f9a1-29e4-45c9-8448-1d6d75f730f8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.204184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f9a1-29e4-45c9-8448-1d6d75f730f8 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.204184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 17/34] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:53.204184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589f9a1-29e4-45c9-8448-1d6d75f730f8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.240351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cea95-9d6b-4717-ba5a-df4bf92cc454 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.282109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.282841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.283494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.283494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.296669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fe44c767-c2b8-4a90-adc5-39b384d7cd1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:13:53.302138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:13:53.303227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 18/35] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:13:53.308978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.309875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.309875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.326966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:53.328355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:53.366042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589f9a1-29e4-45c9-8448-1d6d75f730f8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.392628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.412429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.413064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.413344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.413617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.414422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.415992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.416921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 19/36] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:53.418702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.460156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 18/37] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:13:53 2022] POST /identity/v3/projects => generated 301 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:13:53.478220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.548420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32836d5d-e96b-47bd-8746-ebcbbe9821ee None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.588420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.589459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.589459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.589459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.592181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:13:53.592181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.592181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a25b68-573a-42ad-b116-4fef1bee8b2e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.601982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 20/38] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:53.608085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.614454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc625b2b-2159-4622-9488-9e7cba2e6bb6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.615039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc625b2b-2159-4622-9488-9e7cba2e6bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.615524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc625b2b-2159-4622-9488-9e7cba2e6bb6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.617165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 21/39] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:13:53.625526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.626124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.626591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.745661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f65bcd-0241-469a-bf26-ef3ea38e8f17 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.753773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:53.755083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:53.781119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.781862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.782144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.782419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] PATH_INFO: `/v3/users/c24104e3c5bf48be865c100865a4abd3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.783796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] RBAC: Authorizing `identity:get_user(user_id=c24104e3c5bf48be865c100865a4abd3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.785298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.786906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 19/40] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:13:53 2022] GET /identity/v3/users/c24104e3c5bf48be865c100865a4abd3 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:13:53.787672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.822271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 22/41] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:13:53.836296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:53.836407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.854230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:53.899598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf8324-e493-404b-a4f0-2c71071c4a8b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.925343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d71a5-0ae9-4966-ba21-df329f5c0b46 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:53.933701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.934324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.934614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.934881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.935544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.936977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.953562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 20/42] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 11 13:13:53 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:13:53.960833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:53.961511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:53.961791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:53.962063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:53.963247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:53.964835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:53.966668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 23/43] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:53.973025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:54.083288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:54.087260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd5cc-ecd1-4298-a38b-680fabe16108 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:54.089280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:54.129111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:54.129743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:54.129939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:54.130214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:54.130969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.domain.id=default, user.id=c24104e3c5bf48be865c100865a4abd3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:13:54.131265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:54.132831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:54.144186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 21/44] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Oct 11 13:13:53 2022] GET /identity/v3/role_assignments?user.id=c24104e3c5bf48be865c100865a4abd3&scope.domain.id=default&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:13:54.195504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:54.198431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda5c1-cefc-4c76-af70-70d07626c258 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:54.234071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:54.234727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:54.235006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:54.235280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:54.236489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:54.237971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47669b72-9804-4924-bce5-5c7969bab2d9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:54.239632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 24/45] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:13:54.245656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:54.355520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a586036-891d-4fc7-b18b-90fc1cb7411e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:54.389676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:54.390237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:54.390517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:54.390786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:54.391437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:54.392916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4624aa-5792-40f7-b40e-1458534d81a4 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:54.421188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 22/46] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:13:54 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:13:56.539261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677665-cddf-4263-a5e3-5afacffab9e4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:56.539798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677665-cddf-4263-a5e3-5afacffab9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:56.539798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677665-cddf-4263-a5e3-5afacffab9e4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:56.541237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 25/47] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:56.547522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:56.548229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:56.548481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:56.571369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:56.572926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:56.587754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:56.647037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 23/48] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:56.921658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e677665-cddf-4263-a5e3-5afacffab9e4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:57.035267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e677665-cddf-4263-a5e3-5afacffab9e4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:57.069961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:57.070636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:57.070912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:57.071176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:57.072422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:57.073867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b03a1-92b3-4cb6-87dd-4fc9e9500efe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:57.075418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 26/49] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:57.081904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:57.191184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5864-97dd-4993-8011-4079cc18849d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:57.226027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:57.226650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:57.226929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:57.227196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:57.227843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:57.229304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71d42f-d5e8-40c4-92e7-6d06e786143f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:57.253193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:57.260690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 24/50] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 11 13:13:57 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:13:59.379731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede737-aa36-4d6d-8ab1-268cb9e7e838 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:59.380337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede737-aa36-4d6d-8ab1-268cb9e7e838 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:59.380337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede737-aa36-4d6d-8ab1-268cb9e7e838 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:59.381688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 27/51] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:13:59.387222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:59.387630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:59.387742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:59.408219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:13:59.409409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:13:59.484436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 25/52] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:13:59.749415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aede737-aa36-4d6d-8ab1-268cb9e7e838 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:13:59.797495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:13:59.864848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aede737-aa36-4d6d-8ab1-268cb9e7e838 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:13:59.900220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:13:59.900856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:13:59.901142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:13:59.901413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:13:59.902645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:13:59.904104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827c11e-7bc0-428f-a66d-e509cf80f8dc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:13:59.905723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 28/53] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:13:59.911536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:00.020139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073bede-3d0e-4ec9-8b90-062316523c93 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:00.049272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:00.055891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:00.056540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:00.056823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:00.057101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:00.057760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:00.059216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44263-5582-4508-b7be-39466775c044 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:00.086362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 26/54] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:13:59 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:03.176363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128030-e1b4-4cfd-9a1e-ce1c79168009 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.177381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128030-e1b4-4cfd-9a1e-ce1c79168009 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.177852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128030-e1b4-4cfd-9a1e-ce1c79168009 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.179527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 29/55] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:03.185302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f222-188d-45ae-81af-cae12e419e6a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.186179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f222-188d-45ae-81af-cae12e419e6a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.186683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f222-188d-45ae-81af-cae12e419e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.217091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-65d7f222-188d-45ae-81af-cae12e419e6a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:03.218617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-65d7f222-188d-45ae-81af-cae12e419e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:03.295786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083dd99-8608-4b5c-ab92-308ad646fc52 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.296427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083dd99-8608-4b5c-ab92-308ad646fc52 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.296919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083dd99-8608-4b5c-ab92-308ad646fc52 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.298464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 30/56] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:03.303885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036c23-1d79-4a72-96d6-89d6886d44c3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.304620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036c23-1d79-4a72-96d6-89d6886d44c3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.305084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036c23-1d79-4a72-96d6-89d6886d44c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.345356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-09036c23-1d79-4a72-96d6-89d6886d44c3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:03.346594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-09036c23-1d79-4a72-96d6-89d6886d44c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:03.452422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 27/57] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:03.486387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 31/58] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:03.535884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1187-272c-4ded-959b-179f3a180610 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.536254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1187-272c-4ded-959b-179f3a180610 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.536738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1187-272c-4ded-959b-179f3a180610 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.538284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 28/59] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:03.543670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.543961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.544276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:03.569617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:03.570894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:03.609951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:03.661408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 32/60] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:03.804385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e1187-272c-4ded-959b-179f3a180610 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:03.921987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e1187-272c-4ded-959b-179f3a180610 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:03.948538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:03.948727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:03.984608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:03.985276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:03.985560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:03.985839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.006546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:04.013016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:04.013841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:04.014602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 29/61] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:04.025000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.106496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc535a06-e7b2-4db5-b2da-72af48b256a5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.139396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.140040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.140351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.140623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.140872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20dfc3-87a9-4492-bd70-d9ae44ee62c0 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.151228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:04.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:04.156971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:04.157710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 33/62] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:04.158345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.177001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.177628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.177912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.178189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.178865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:04.179179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.180691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b7aa-61a6-4b11-bef9-1ed62d7a6a1f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.182667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.192693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 30/63] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.193899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da3cf-c1b3-4c8d-8589-b11255e118e6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.194187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da3cf-c1b3-4c8d-8589-b11255e118e6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.194460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da3cf-c1b3-4c8d-8589-b11255e118e6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.195717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 31/64] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:04.196766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e72cac-82f3-4f72-a54c-86374a389581 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.197051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e72cac-82f3-4f72-a54c-86374a389581 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.197328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e72cac-82f3-4f72-a54c-86374a389581 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.201708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 32/65] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:04.201708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e72cac-82f3-4f72-a54c-86374a389581 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.239993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.262753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c674ba-a2a0-4942-9925-7b0124a411ef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.298668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.299311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.299470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e72cac-82f3-4f72-a54c-86374a389581 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.299591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.299865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.310722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:04.315553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:04.316501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:04.317257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 34/66] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:04.317879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.335434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.336088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.336408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.336679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.337334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:04.337640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.339036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a452c4-a8fa-4987-bf02-8fbdc8bdc576 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.347196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 33/67] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.348233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a258-c4b2-49fc-a845-f2e956b6133b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.348517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a258-c4b2-49fc-a845-f2e956b6133b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.348792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a258-c4b2-49fc-a845-f2e956b6133b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.367075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a3b8a258-c4b2-49fc-a845-f2e956b6133b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:04.368235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a3b8a258-c4b2-49fc-a845-f2e956b6133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:04.406032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.422572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e19d1-c877-450e-99b8-1aac4186db0c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.448144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 34/68] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:04.448144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.448144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.448144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.455959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.456726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.457044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.457392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.465341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:04.466697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:04.471419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.473008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.474914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.475586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.490948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.535089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 35/70] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:04.535846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.583208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33afe8-7f35-43f2-8c45-9bc62a849676 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.620895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.621874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.622763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.623251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.624453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:04.625129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.627351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.634387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775171-8443-4275-9ff9-17c2959e6680 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.638770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 36/71] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.639787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.643880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.671089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.671824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.672251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.672679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] PATH_INFO: `/v3/users/c24104e3c5bf48be865c100865a4abd3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.674024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=c24104e3c5bf48be865c100865a4abd3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.675586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.677519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 36/72] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/users/c24104e3c5bf48be865c100865a4abd3 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.678470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.708185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.752373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70d47-210c-4386-bbb2-1faa526290fd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.786104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68ade-9d41-44df-9f54-52454b66d8b9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.787927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.788764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.789229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.789509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.790653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.792166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.793827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 37/73] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.794944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.827130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.827771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.828052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.828357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.829482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.830917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.832519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 37/74] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.833272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.866070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.894714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008082aa-63ff-4462-811b-bea93a137f70 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.926886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:04.931842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.932511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.932790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.933062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.933152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db81ce7-a66a-4e69-993e-2c97a153c1f2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:04.934215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:04.935660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:04.937354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:04.938072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:04.973178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:04.973856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:04.974246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:04.974417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:04.984921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:04.989801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:04.990747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:04.991505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 38/76] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:04.992371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.042405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a681a-158f-4270-8c29-78d7f8ff887f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.077272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.077947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.078231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.078509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/394c672b73ca4f16aca0a84b753bfd89/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.080482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, user_id=394c672b73ca4f16aca0a84b753bfd89, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.081951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.090786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772f81-ba6b-40cd-b32e-7ab5e70cd8be None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.091936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:05.102371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 39/77] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:04 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/394c672b73ca4f16aca0a84b753bfd89/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:05.102371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.139009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.140107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.140594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.141017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.143010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.145563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.148350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 39/78] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:05.149338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.180336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:05.216474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0dd7-68e3-49ff-a081-d023517ca570 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.254615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.255647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.255939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.256250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.257692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.259191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.259852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7c54-5d4c-4d28-ab69-f9c6a3028f85 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.265888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 40/79] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:05 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:05.265888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.304442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.305137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.305417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.305693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/c24104e3c5bf48be865c100865a4abd3/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.307648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, user_id=c24104e3c5bf48be865c100865a4abd3, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.309169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.324373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 40/80] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:05 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/c24104e3c5bf48be865c100865a4abd3/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:05.325063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.348398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:05.398422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7771f-f131-4449-8a9d-416f329cdeb3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.426880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:05.445255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.446427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.446964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.447475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.448313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:05.448730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.449740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e90b3-537b-4db3-8f3e-91d62fc4e2fa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.450366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.466876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:05.466876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.503500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.504592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.505001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.506433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.507122 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.509515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.537392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 41/82] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:05 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:05.539111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.593299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c91c-e0e3-4dca-9683-6b0c4ecb6329 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.632957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:05.641832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.642776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.643062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.643339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/394c672b73ca4f16aca0a84b753bfd89/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.646979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, user_id=394c672b73ca4f16aca0a84b753bfd89, role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.649802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.670242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 42/83] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:05 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/394c672b73ca4f16aca0a84b753bfd89/roles/314f2033c1db4762926d37c1532eb2f1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:05.692679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dce59-873d-4b17-bc1e-0db113dc4eaf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.741458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.742498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.742975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.743486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.746287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.748133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.750317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 42/84] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:05 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:05.756334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.888814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9fb8-40fd-4a77-bdd6-c0e0df8faeda None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:05.925822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:05.926400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:05.926683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:05.926957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] PATH_INFO: `/v3/projects/5a40aa49f2574a208524bda838a462b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:05.928081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] RBAC: Authorizing `identity:get_project(project_id=5a40aa49f2574a208524bda838a462b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:05.929615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836e1f8-46da-42e3-a9eb-543c0bdba5ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:05.931270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:05 2022] GET /identity/v3/projects/5a40aa49f2574a208524bda838a462b4 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:05.937071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:05.954219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:06.053864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaf323-911d-4be2-9477-6173263e50bf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:06.089247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:06.090274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:06.090710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:06.091126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] PATH_INFO: `/v3/projects/5a40aa49f2574a208524bda838a462b4/users/394c672b73ca4f16aca0a84b753bfd89/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:06.094352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a40aa49f2574a208524bda838a462b4, user_id=394c672b73ca4f16aca0a84b753bfd89, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:06.095957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f753-77cd-4b8c-8f6d-7bc9a1722e08 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:06.117051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 43/86] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:05 2022] PUT /identity/v3/projects/5a40aa49f2574a208524bda838a462b4/users/394c672b73ca4f16aca0a84b753bfd89/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:07.989917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108f53-0aa4-49a8-a1d0-ccd659e08416 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:07.990950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108f53-0aa4-49a8-a1d0-ccd659e08416 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:07.991433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108f53-0aa4-49a8-a1d0-ccd659e08416 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:07.993160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 44/87] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:07.998578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:07.999394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:07.999867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.008756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61129-300f-4f13-9cc1-a564900e4848 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.009300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61129-300f-4f13-9cc1-a564900e4848 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.009809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61129-300f-4f13-9cc1-a564900e4848 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.011305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 45/88] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:08.016228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459c81-4f8a-4301-875f-652591ab9398 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.016738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459c81-4f8a-4301-875f-652591ab9398 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.017210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459c81-4f8a-4301-875f-652591ab9398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.039808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:08.041560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:08.045914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-62459c81-4f8a-4301-875f-652591ab9398 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:08.049798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-62459c81-4f8a-4301-875f-652591ab9398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:08.167255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:08.190502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:08.196534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 46/89] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:08.198690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dc84f-180c-4e33-b5f9-474aadebb077 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.199351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dc84f-180c-4e33-b5f9-474aadebb077 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.201166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dc84f-180c-4e33-b5f9-474aadebb077 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.202986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 47/90] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:08.205231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 44/91] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:08.221984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.222522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.222973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.253169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:08.254669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:08.337910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 48/92] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:08.538111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:08.607178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:08.685447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5606d-3487-4ae0-9c80-04fa99365767 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:08.744460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:08.745496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.746001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.746483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.764028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:08.776023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:08.776923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:08.777698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:08.780836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 45/93] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:08.786775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:08.838157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d6429-b2d0-4560-81d3-edb3240da062 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:08.877338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:08.880335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:08.917578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:08.918241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.918517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.918786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.938616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:08.943609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b83c2-fa6d-47bf-8c32-72a53d17199f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:08.951913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:08.952817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:08.953582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 49/94] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:08.957048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:08.985100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:08.985808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:08.986089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:08.986422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:08.987108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:08.987418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:08.988918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:08.998017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 46/95] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:08.998717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.107659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7344880-c1e6-401b-ae5a-f95c7e5becd6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.128404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.151256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.151882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.152187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.152463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.153145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:09.153454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.154850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.171636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 47/96] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:09.177052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.185201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c82971-74aa-4bed-b17b-2c4de357518d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.255608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.256359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.256642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.256913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.268064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:09.273473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:09.274353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:09.275120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 50/97] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:09.275854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.330205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.359131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7526a-09f0-4c3d-8464-d23d0f784a9f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.390964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcb1e3-bdfa-4d39-b271-67158b9a8b85 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.401285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.402615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.403318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.403770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] PATH_INFO: `/v3/users/c24104e3c5bf48be865c100865a4abd3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] RBAC: Authorizing `identity:get_user(user_id=c24104e3c5bf48be865c100865a4abd3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.406920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.409706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 48/98] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/users/c24104e3c5bf48be865c100865a4abd3 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:09.410986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.426383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.427018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.427298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.427564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.428703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.430206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccff05-0475-489c-9299-8dfaba1ffbff None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.431776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 51/99] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:09.433037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e51da-b6bc-4cd9-b62b-8ecfb86bdaf3 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.433377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e51da-b6bc-4cd9-b62b-8ecfb86bdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.433644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e51da-b6bc-4cd9-b62b-8ecfb86bdaf3 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.434827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 52/100] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:09.435822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566429e-93f5-4832-b7c1-868c25f78428 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.436312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566429e-93f5-4832-b7c1-868c25f78428 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.436520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566429e-93f5-4832-b7c1-868c25f78428 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.437789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 53/101] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:09.438422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566429e-93f5-4832-b7c1-868c25f78428 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.467078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.523076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7003120-5dfb-4c87-b461-b8b80802d263 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.537203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566429e-93f5-4832-b7c1-868c25f78428 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.544249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.557457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.558087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.558372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.558647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.559327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:09.559637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.561078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0216e-1be9-416d-9742-3bea5747cf60 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.569412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 49/102] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:09.570677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94060-f8a8-470f-9dd4-3d911db813b9 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.570781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.570967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94060-f8a8-470f-9dd4-3d911db813b9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.571229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b94060-f8a8-470f-9dd4-3d911db813b9 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.571439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.571721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.571992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.572732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 50/103] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:09.573214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b94060-f8a8-470f-9dd4-3d911db813b9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.644179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.645585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bb4aa-2ec4-4bc5-838c-8d79997be2c7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.673464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b94060-f8a8-470f-9dd4-3d911db813b9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:09.686152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.708266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:09.708889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.709157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.709430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.740059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 54/104] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:09.741458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3b97-2849-4cbf-a119-9a2864d12ea5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.741740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3b97-2849-4cbf-a119-9a2864d12ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.742042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3b97-2849-4cbf-a119-9a2864d12ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.753723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.759903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5a0d3b97-2849-4cbf-a119-9a2864d12ea5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:09.761135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a0d3b97-2849-4cbf-a119-9a2864d12ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:09.780755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:09.782225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:09.823448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 55/105] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:09.824698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02772977-b1dd-41e9-a659-411c846a8d91 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.824979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02772977-b1dd-41e9-a659-411c846a8d91 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.825251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02772977-b1dd-41e9-a659-411c846a8d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.856222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-02772977-b1dd-41e9-a659-411c846a8d91 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:09.857209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02772977-b1dd-41e9-a659-411c846a8d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:09.882519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 51/106] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:09.883447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:09.900827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:09.930667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 56/107] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:09.932868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:09.933596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:09.934051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:09.960716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:09.962161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:09.988435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52f8a-4713-4f29-844e-15523d066009 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.024300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.024891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.025103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.025381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.026562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.028558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.029820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 52/108] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:09 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.030606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.036609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.045796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 57/109] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:10.046878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.148740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.164095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99423eb-a6d6-4f51-94ad-a277440bbdf8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.170285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf989a-ff01-4d66-8883-ba59c295c564 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.204500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.205136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.205412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.205682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.206787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.208251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.208995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.209876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 53/110] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.210331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.210758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.211060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.211473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.212787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.215708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.217683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 58/111] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.218602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.291417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.317681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850558d0-bce8-4819-8dfe-ed1b2e59cd09 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.330539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cc65-dadf-49a0-9987-deda2c339513 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.356719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.357343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.357625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.357901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.362936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.363567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.363844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.364130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.373799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:10.375213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:10.378629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:10.379481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:10.380291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 59/112] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:10.380913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.431015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.432512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.477135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483c08-7f1e-4892-b067-b756470e2ec4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.499366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.509811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.510453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.510737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.511018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.521215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:10.526409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:10.527497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 60/113] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:10.528465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.529215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 54/114] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:10.529556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.577835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.627185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56563da5-3582-4f8a-952b-32817e99a651 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.627504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90aa69-7c0c-4ea9-8656-e529b11a3428 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.660310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.660928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.661211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.661519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.662129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.662243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] RBAC: Adding query filter params (role.id=d121dc77eff143a186cdd79b2ce341b4, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1, user.id=394c672b73ca4f16aca0a84b753bfd89) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:10.662551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.662749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.663032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.663302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.664054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.664241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1, user.id=c24104e3c5bf48be865c100865a4abd3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:10.664521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.665955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.669806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 55/115] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/role_assignments?user.id=394c672b73ca4f16aca0a84b753bfd89&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.670592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.673039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 61/116] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/role_assignments?user.id=c24104e3c5bf48be865c100865a4abd3&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.673867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.759074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.820453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4191b48-6961-4a5b-94e0-f1dff7c2c8e7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.831703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64574a-24ab-4544-a82a-2c306c444c8e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:10.839578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:10.880094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.880746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.881023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.881294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.881954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:10.882068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:10.882253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.882785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:10.883066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:10.883340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:10.883686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.884526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:10.885975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:10.887761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 56/117] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.889258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:10.893565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 62/118] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:10.893846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.002514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aff2c-4c76-40df-861e-91c3b7c409e1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.004781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e4f8c-98ec-41d6-bc9c-ebb2debbcb1f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.014657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.035645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.036284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.036560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.036830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.037499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:11.037825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.039219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.039686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.040354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.040635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.040911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.042030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.043473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.045318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 63/119] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.045987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.054591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 57/120] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.054591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.092942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.148054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bae56a-a1e7-4f35-840c-091b06b21863 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.150066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492df8-5fbc-483c-af31-878f5d581aea None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.180986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.181599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.181879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.182148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.182800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.183048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.183685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.183967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.184273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] PATH_INFO: `/v3/users/04cd442908c64f8683a4fd4460488b38` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.184361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.199911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] RBAC: Authorizing `identity:get_user(user_id=04cd442908c64f8683a4fd4460488b38)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.201393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.203027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 58/121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/users/04cd442908c64f8683a4fd4460488b38 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.203798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.228220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 64/122] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:14:11 2022] POST /identity/v3/projects => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:11.228220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.248009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.330575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116178c-5e3f-4bd0-b306-084e5191b6dd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.335305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.338627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c544da-0e6b-411e-a2c3-6b39ea9d4bba None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.369269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.370016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.370283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.370558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] PATH_INFO: `/v3/users/394c672b73ca4f16aca0a84b753bfd89` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.371758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=394c672b73ca4f16aca0a84b753bfd89)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.373290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.375100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 65/123] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/users/394c672b73ca4f16aca0a84b753bfd89 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.375799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.385264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.386699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.386978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.387253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.388001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] RBAC: Adding query filter params (role.id=314f2033c1db4762926d37c1532eb2f1, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1, user.id=394c672b73ca4f16aca0a84b753bfd89) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:11.388343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.390535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.397759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 59/124] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/role_assignments?user.id=394c672b73ca4f16aca0a84b753bfd89&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=314f2033c1db4762926d37c1532eb2f1 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.398031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.500069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.505980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5887b99-1bd7-466f-8e8f-508027029ea6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.535683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667780da-88c4-4be7-9649-44a44ffd276c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.545989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.546490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.546768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.547040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.548248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.549860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.552030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 66/125] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.558018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.585034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.586064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.586546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.586973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.630162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.677510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.679228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.680315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a49-d03e-4545-85d2-f73a25b70a71 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.718309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.718995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.719273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.719544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/04cd442908c64f8683a4fd4460488b38/roles/0d1fc9612ff54371b3b4106b28cacff2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.721470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, user_id=04cd442908c64f8683a4fd4460488b38, role_id=0d1fc9612ff54371b3b4106b28cacff2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.723414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.748142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 67/126] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:11 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/users/04cd442908c64f8683a4fd4460488b38/roles/0d1fc9612ff54371b3b4106b28cacff2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:14:11.790786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 60/127] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:11.796364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:11.828268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:11.912025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6692b-175d-4166-bf84-81553be8a8c0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:11.949074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:11.949712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:11.950091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:11.950363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] PATH_INFO: `/v3/projects/5a40aa49f2574a208524bda838a462b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:11.951460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] RBAC: Authorizing `identity:get_project(project_id=5a40aa49f2574a208524bda838a462b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:11.952947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd23ab-bbcd-4b07-99e6-9c9b65e8c473 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:11.954790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 68/128] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/projects/5a40aa49f2574a208524bda838a462b4 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:11.960445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:12.023556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:12.062637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcaebd-9180-4e63-9c70-6d391a136eae None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:12.098025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:12.098654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:12.098940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:12.099219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:12.099973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] RBAC: Adding query filter params (role.id=d121dc77eff143a186cdd79b2ce341b4, scope.project.id=5a40aa49f2574a208524bda838a462b4, user.id=394c672b73ca4f16aca0a84b753bfd89) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:12.100317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:12.101936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2597a-6f3d-4d90-8af0-25d46db9a7b8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:12.116197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 61/129] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:11 2022] GET /identity/v3/role_assignments?user.id=394c672b73ca4f16aca0a84b753bfd89&scope.project.id=5a40aa49f2574a208524bda838a462b4&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:13.450310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0bf6-96d4-484c-86d5-d89d4320828e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:13.450846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0bf6-96d4-484c-86d5-d89d4320828e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:13.450846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0bf6-96d4-484c-86d5-d89d4320828e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:13.452233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 69/130] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:13.457676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665cb5-32b8-4642-bd55-956529da4432 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:13.458089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665cb5-32b8-4642-bd55-956529da4432 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:13.458218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665cb5-32b8-4642-bd55-956529da4432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:13.477151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a9665cb5-32b8-4642-bd55-956529da4432 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:13.478344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a9665cb5-32b8-4642-bd55-956529da4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:13.551676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 62/131] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:13.824369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f0bf6-96d4-484c-86d5-d89d4320828e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:13.845154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d10a8-4b91-431d-85d1-263a00f5aeeb None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:13.845439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d10a8-4b91-431d-85d1-263a00f5aeeb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:13.845713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d10a8-4b91-431d-85d1-263a00f5aeeb None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:13.847266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 63/132] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:13.852187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:13.852481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:13.852752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:13.872543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:13.873758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:13.918290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:13.926456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f0bf6-96d4-484c-86d5-d89d4320828e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:13.926943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:13.940868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 64/133] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:13.959884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:13.960576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:13.960915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:13.961186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:13.962292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:13.963735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:13.965366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 70/134] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:13.971053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:14.065847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa5246-777a-484e-962a-925224bf5e28 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:14.100046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:14.100616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:14.100903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:14.101175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:14.101888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:14.103648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:14.124853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 65/135] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:13 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:14.201530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:14.311851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11dfa2-c719-4b88-a99d-693ef46d0d4a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:14.345049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:14.345692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:14.345971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:14.346249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:14.356436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:14.361212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:14.362074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:14.363051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 71/136] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:14.368364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:14.421942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:14.478255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e658ef-2b9a-42d7-9bee-33a3e26def2a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:14.514382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:14.515040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:14.515386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:14.515583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:14.516296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:14.516605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:14.517985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:14.528846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 66/137] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:14.534375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:14.561792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:14.642915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db073-62c7-45d4-b6d9-65b809d13ac9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:14.677166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:14.677872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:14.678153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:14.678477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] PATH_INFO: `/v3/users/04cd442908c64f8683a4fd4460488b38` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:14.679708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] RBAC: Authorizing `identity:get_user(user_id=04cd442908c64f8683a4fd4460488b38)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:14.681223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:14.683119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 72/138] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:14 2022] GET /identity/v3/users/04cd442908c64f8683a4fd4460488b38 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:14.688843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:14.788433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62f64-9cc9-4621-978d-e3b397012831 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:14.800476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:14.822642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:14.823263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:14.823544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:14.823818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:14.894919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:14.896355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:14.995387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 67/139] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:15.001029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:15.097378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:15.108493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21acf7-5dfb-4230-a692-f4f5987e2026 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:15.143476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:15.144142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:15.144441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:15.144718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:15.145910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:15.147652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f602-b4d8-404f-8f0c-5a3283b9ef40 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:15.149733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 73/140] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:14 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:15.155264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:15.163552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:15.255401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2856e6-ddbf-4291-81d8-c573e29a9d1c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:15.289476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:15.290090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:15.290366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:15.290635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:15.291367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] RBAC: Adding query filter params (role.id=0d1fc9612ff54371b3b4106b28cacff2, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1, user.id=04cd442908c64f8683a4fd4460488b38) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:15.291670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:15.293119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d681-4a25-404a-9a59-8e8347f2faa6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:15.299291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 68/141] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:15 2022] GET /identity/v3/role_assignments?user.id=04cd442908c64f8683a4fd4460488b38&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=0d1fc9612ff54371b3b4106b28cacff2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:16.347652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76697af1-dcbc-464f-9586-c70f38ef2378 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.348276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76697af1-dcbc-464f-9586-c70f38ef2378 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.348276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76697af1-dcbc-464f-9586-c70f38ef2378 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.349559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 74/142] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:16.354849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca384-a7ee-4d6b-90c9-8b1709fcea66 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.355258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca384-a7ee-4d6b-90c9-8b1709fcea66 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.355425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca384-a7ee-4d6b-90c9-8b1709fcea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.374629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-007ca384-a7ee-4d6b-90c9-8b1709fcea66 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:16.375820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-007ca384-a7ee-4d6b-90c9-8b1709fcea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:16.438497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:16.448270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 69/143] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:16.576394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfd39-f52a-414a-af2a-88d9e4f28b0b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.576790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfd39-f52a-414a-af2a-88d9e4f28b0b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.576894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccfd39-f52a-414a-af2a-88d9e4f28b0b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.578461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 75/144] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:16.583509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.583802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.584081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.602777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:16.603883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:16.644323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639f9c-aae9-4ef9-ac3a-37e979ac2ed5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.646180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639f9c-aae9-4ef9-ac3a-37e979ac2ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.646180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639f9c-aae9-4ef9-ac3a-37e979ac2ed5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.646180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 76/145] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:16.651723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.652010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.652318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.670888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:16.672032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:16.674108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 70/146] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:16.707155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:16.748833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 77/147] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:16.809772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb18e2-60ae-4b11-aa10-b151c675b154 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:16.838496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:16.845460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:16.846103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:16.846379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:16.846676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:16.857084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:16.861932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:16.862882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:16.863816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 71/148] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:16.870753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:16.930345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:16.961149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:16.974914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724665f-1b70-4523-b312-8267cc7d6eb6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.010710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.011328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.011604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.011874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.012564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:17.012886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.014292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.023356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 78/149] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.024253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.032547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daeccbb-4b7b-4399-8e91-a1b3298a0daa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.065059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.065692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.065963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.066219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.076244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:17.081086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:17.081977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:17.082741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 72/150] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:17.083439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.127786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2542-73a7-4fb0-82aa-bd8a112a5108 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.133721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.160459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.161099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.161409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.161669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.162798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.164408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.165978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 79/151] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.166726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.183955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf08ab-04c1-4f63-ad49-84186a9aab59 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.188427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.219120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.220031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.220347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.220622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] PATH_INFO: `/v3/users/9cf19f25d01241a7a197e05cfe558f02` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.233960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] RBAC: Authorizing `identity:get_user(user_id=9cf19f25d01241a7a197e05cfe558f02)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.235414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.237049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 73/152] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/users/9cf19f25d01241a7a197e05cfe558f02 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.239254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.267585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44876-d9c3-4bb3-8b9d-434b2c308e7e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.301351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.301951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.302218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.302488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.303144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:17.303443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.304875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.312456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 80/153] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.315141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.334478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948ff3-b7da-42a2-bc0d-04923af9bf52 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.344154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.367486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.368154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.368381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.368646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.369290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.370742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.391449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 74/154] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:17 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:17.393073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.395483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.419881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9effb9f-576c-4aad-896a-3f32e62531c4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.460102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.460918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.461197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.461472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.462643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.464158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.466083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 81/155] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.471881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.508879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcab9-9da9-48e8-af08-b1b1ae4b0d14 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.545437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.546083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.546355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.546622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] PATH_INFO: `/v3/users/9cf19f25d01241a7a197e05cfe558f02` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.547783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] RBAC: Authorizing `identity:get_user(user_id=9cf19f25d01241a7a197e05cfe558f02)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.549306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.555728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 75/156] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/users/9cf19f25d01241a7a197e05cfe558f02 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:17.556698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.591441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a80062-83fd-4861-b8d7-b195e3d38968 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.591922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.633293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.633970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.634242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.634509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/9cf19f25d01241a7a197e05cfe558f02/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.636717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, user_id=9cf19f25d01241a7a197e05cfe558f02, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.638960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.659719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 82/157] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:17 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/9cf19f25d01241a7a197e05cfe558f02/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:14:17.677110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b659b9-4a49-4d60-9005-e97bcbadd23a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.717559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.718222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.718502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.718774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.720014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.721578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed09bd-21c5-433e-ae04-a25a398875e8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.724349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 76/158] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:17 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:17.731722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:17.747150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:17.850493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cc5e-66d7-4c3f-9784-2fc860226e72 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:17.885209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:17.885937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:17.886262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:17.886534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/9cf19f25d01241a7a197e05cfe558f02/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:17.888710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, user_id=9cf19f25d01241a7a197e05cfe558f02, role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:17.890242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396c9aa-fff7-4215-af66-47ab7a3ba776 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:17.905595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 83/159] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:17 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/9cf19f25d01241a7a197e05cfe558f02/roles/314f2033c1db4762926d37c1532eb2f1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:19.672910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a061-d1b2-4d9b-bca3-5febce69cc69 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.673470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a061-d1b2-4d9b-bca3-5febce69cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.673470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a061-d1b2-4d9b-bca3-5febce69cc69 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.674949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 77/160] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:19.681378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584df77-4b30-4cd1-a5c7-33a37d19613a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.681922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584df77-4b30-4cd1-a5c7-33a37d19613a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.682146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584df77-4b30-4cd1-a5c7-33a37d19613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.702180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5584df77-4b30-4cd1-a5c7-33a37d19613a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:19.703462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5584df77-4b30-4cd1-a5c7-33a37d19613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:19.767288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:19.775546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 84/161] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:19.793946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5a7d-2d2c-4458-8154-758b30904758 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.794235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5a7d-2d2c-4458-8154-758b30904758 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.794502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5a7d-2d2c-4458-8154-758b30904758 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.795987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 78/162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:19.800220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a6c0-927d-45c6-a75a-b643f61af87d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.800509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a6c0-927d-45c6-a75a-b643f61af87d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.800780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a6c0-927d-45c6-a75a-b643f61af87d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.801727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.802129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 85/163] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:19.802430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.802774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.809357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f2797-6be3-4dc9-8e67-84fdf46f273d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:19.809871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f2797-6be3-4dc9-8e67-84fdf46f273d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:19.810330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f2797-6be3-4dc9-8e67-84fdf46f273d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:19.823024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:19.824841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:19.828343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-182f2797-6be3-4dc9-8e67-84fdf46f273d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:19.829488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-182f2797-6be3-4dc9-8e67-84fdf46f273d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:19.898451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 79/164] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:19.915315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 86/165] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:20.045076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.055761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6d019-fa7d-4523-a60a-8eeefeae1812 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.056102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6d019-fa7d-4523-a60a-8eeefeae1812 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.056775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6d019-fa7d-4523-a60a-8eeefeae1812 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.058616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 87/166] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:20.064908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.065193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.065470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.077599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.092480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:20.093551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:20.161205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e4e5c-fa9d-488f-860a-d9a961ab4864 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.169706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 88/167] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:20.173061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.203061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.203947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.204420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.204892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.215092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:20.220040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:20.221067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:20.221840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 80/168] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:20.222512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.237725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.286006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e2f94-e082-4df8-bff8-cb2e7604b789 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.318887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.321931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.322578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.322855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.323121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.324522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.325988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.327561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 89/169] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:20.327561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.332207 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8f73-967d-4a8c-b47f-427087bcae04 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.367952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.368609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.368880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.369200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.379921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:20.385108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:20.385971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:20.386746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 81/170] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:20.389240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.432249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e14cb0-5e32-4f2e-9462-f5a69976b393 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.457506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.465790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.466405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.466676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.466940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.467604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:20.467899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.469321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.477266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 90/171] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:20.478043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.490847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16bfe8-7ffb-4e9a-866f-67d62d867233 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.526002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.526607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.526878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.527142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.527783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.529260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.550195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 82/172] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:20 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:20.550997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.561113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.578582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a64305-f22b-42b1-8fb4-4d98cd403829 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.618942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.619570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.619848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.620147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.620826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:20.621127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.622540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.631794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 91/173] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:20.632771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.681896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197981c7-9793-4e42-800c-4abd55e129c2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.688762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.722923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.723597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.724156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.724437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.736826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:20.743315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:20.744333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:20.745121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 83/174] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:20.745769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.748276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8e07c-bd7f-46ce-a98f-9369076545bc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.787934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.788686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.788935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.789447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] PATH_INFO: `/v3/users/271a314c0b184c3b9b9c7bcede4bb686` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.806305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] RBAC: Authorizing `identity:get_user(user_id=271a314c0b184c3b9b9c7bcede4bb686)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.807831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.809905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 92/175] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/users/271a314c0b184c3b9b9c7bcede4bb686 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:20.810820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.821557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.864231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610be6f-2a62-4579-aa88-0c04ad32788f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.907826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.908512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.908794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.909071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] PATH_INFO: `/v3/users/9cf19f25d01241a7a197e05cfe558f02` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.910296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] RBAC: Authorizing `identity:get_user(user_id=9cf19f25d01241a7a197e05cfe558f02)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.913016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:20.914884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 84/176] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/users/9cf19f25d01241a7a197e05cfe558f02 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:20.915849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:20.936970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844884b-1c45-4244-92e6-31a992537949 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:20.948906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:20.987951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:20.989200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:20.990065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:20.990805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:20.991740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:20.992277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:20.994148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.006159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 93/177] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:21.007829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.030320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39a6b-34b7-413a-8131-4ec7ef932f00 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.067037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.067814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.068223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.068615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.070278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.071613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.073495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 85/178] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:20 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:21.078519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.079228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.133628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666339-e258-4fb4-9b5b-aeb07af9812e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.169711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.170624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.171110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.171745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.212421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b5c1-dce1-4476-b784-35e53d099b50 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.216576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.250384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.251022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.251295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.251565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] PATH_INFO: `/v3/users/9cf19f25d01241a7a197e05cfe558f02` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.252741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] RBAC: Authorizing `identity:get_user(user_id=9cf19f25d01241a7a197e05cfe558f02)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.254410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.256247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 86/179] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/users/9cf19f25d01241a7a197e05cfe558f02 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:21.257139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.258697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.261663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.328667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.366954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640aa00-605f-40a2-aeb3-e468e34d7dfd None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.388320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 94/180] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:21.389032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.413984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.415634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.416152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.416866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/271a314c0b184c3b9b9c7bcede4bb686/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.420302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, user_id=271a314c0b184c3b9b9c7bcede4bb686, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.421679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.441530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 87/181] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:21 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/271a314c0b184c3b9b9c7bcede4bb686/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:21.441530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.496001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.555695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961241-b3fe-4c88-a60b-094d11a8ab71 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.590193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010b7ff-da6d-46c3-a340-481e316e0239 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.602321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.606803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.607478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.607761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.608038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.609272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.610895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.612953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 88/182] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:21.618984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.680062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.681020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.681438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.681880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.744389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae1ce-e260-4b04-850e-f7aa59983f41 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:21.759373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.761372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.783787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:21.785159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:21.785159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:21.785159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:21.785885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff, user.id=9cf19f25d01241a7a197e05cfe558f02) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:21.786261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:21.787790 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:21.796196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 89/183] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/role_assignments?user.id=9cf19f25d01241a7a197e05cfe558f02&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:21.841843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:21.891463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 95/184] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:21.897894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:21.977579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:22.019281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910607-860a-4d3f-9ac1-2f0566da4ad2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:22.056655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315100-280d-43d7-bbf9-557e439058de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:22.057314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315100-280d-43d7-bbf9-557e439058de None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:22.057593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315100-280d-43d7-bbf9-557e439058de None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:22.057866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315100-280d-43d7-bbf9-557e439058de None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:22.059018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315100-280d-43d7-bbf9-557e439058de None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:22.061308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315100-280d-43d7-bbf9-557e439058de None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:22.063408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 90/185] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:21 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:22.069267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:22.179100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81e38-cb4d-415b-9b54-217eecd917ba None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:22.214423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:22.215419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:22.215907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:22.216396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:22.217427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] RBAC: Adding query filter params (role.id=314f2033c1db4762926d37c1532eb2f1, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff, user.id=9cf19f25d01241a7a197e05cfe558f02) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:22.217936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:22.219565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcbd5-059f-48e6-af5f-bd754c2ad43d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:22.227950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 96/186] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:22 2022] GET /identity/v3/role_assignments?user.id=9cf19f25d01241a7a197e05cfe558f02&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=314f2033c1db4762926d37c1532eb2f1 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:22.842352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af20a7-b5b5-4253-aad3-da5892d1604b None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:22.842864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af20a7-b5b5-4253-aad3-da5892d1604b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:22.842864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af20a7-b5b5-4253-aad3-da5892d1604b None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:22.844407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 91/187] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:22.849734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484b147-6293-4d52-8b7f-1fea6b58195a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:22.850161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484b147-6293-4d52-8b7f-1fea6b58195a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:22.850257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484b147-6293-4d52-8b7f-1fea6b58195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:22.868804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6484b147-6293-4d52-8b7f-1fea6b58195a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:22.869957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6484b147-6293-4d52-8b7f-1fea6b58195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:22.896142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:22.939092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 97/188] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:23.117975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc25f9-16f9-45b8-bece-9324204984a6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.118309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc25f9-16f9-45b8-bece-9324204984a6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.118501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc25f9-16f9-45b8-bece-9324204984a6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.119843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 92/189] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:23.125194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.125452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.125753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.143946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:23.145157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:23.196563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc25f9-16f9-45b8-bece-9324204984a6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.216481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 98/190] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:23.301883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc25f9-16f9-45b8-bece-9324204984a6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.330284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:23.337391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db495d-0b53-4086-8049-6119502c549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.338035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db495d-0b53-4086-8049-6119502c549c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.338312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db495d-0b53-4086-8049-6119502c549c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.338587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db495d-0b53-4086-8049-6119502c549c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.349256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db495d-0b53-4086-8049-6119502c549c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-93db495d-0b53-4086-8049-6119502c549c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:23.354142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:23.355027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:23.357011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 93/191] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:23.361128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.455602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:23.463781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c321-c47b-4933-ae81-6c1623dd214d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.478678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db495d-0b53-4086-8049-6119502c549c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.501375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.501997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.502273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.502542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.503218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:23.503522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:23.504948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:23.513261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 99/192] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:23.518742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.586358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db495d-0b53-4086-8049-6119502c549c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.623759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.624421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.624697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.624854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9bda8-7dcf-4353-a0de-54d7155ce9dc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.624949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.626033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:23.627532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:23.629141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 94/193] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:23.635056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.663525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.664224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.664481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.664751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] PATH_INFO: `/v3/users/020fb045bc6b4c73a8d725a0c3565cb3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.678283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=020fb045bc6b4c73a8d725a0c3565cb3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:23.679823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:23.681540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 100/194] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:23 2022] GET /identity/v3/users/020fb045bc6b4c73a8d725a0c3565cb3 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:23.686926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.693252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:23.703138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:23.740227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45bb1b-7a0c-49cd-8dc4-5ce19d400761 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.777950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.778676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.778849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.779099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.779751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:23.781226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:23.793367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85437f0-3199-4e5e-8c96-419367298eb1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.803536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 95/195] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:23 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:14:23.836130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:23.836815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.837090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.837335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.838577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:23.840067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8972f-a794-45bb-888a-da255c274334 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:23.842077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 101/196] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:23 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:23.847794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:23.857172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dea8-54da-4f68-8444-4d514ee6740f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.857700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dea8-54da-4f68-8444-4d514ee6740f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.857775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dea8-54da-4f68-8444-4d514ee6740f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.859502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 102/197] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:23.865373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:23.865655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:23.865916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:23.884663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:23.885866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:23.962646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 103/198] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:23.967700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a3b5c-207f-4790-a166-b3280123ff6f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:23.985454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:24.009220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:24.009810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:24.010084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:24.010353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/020fb045bc6b4c73a8d725a0c3565cb3/roles/0d1fc9612ff54371b3b4106b28cacff2` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:24.012288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, user_id=020fb045bc6b4c73a8d725a0c3565cb3, role_id=0d1fc9612ff54371b3b4106b28cacff2)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:24.013721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:24.028370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 96/199] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:23 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/users/020fb045bc6b4c73a8d725a0c3565cb3/roles/0d1fc9612ff54371b3b4106b28cacff2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:24.227000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:24.241128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:24.327629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fb096-5424-49d6-8fd8-ccec001e8824 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:24.361582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:24.362223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:24.362498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:24.362764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:24.373286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:24.378265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:24.379137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:24.379908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 104/200] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:24.385630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:24.489846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5d1f3-7617-4a5a-bce9-b1e25661d636 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:24.525459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:24.526082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:24.526355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:24.526621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:24.527292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:24.527594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:24.528990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:24.537853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 97/201] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:24.543230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:24.620716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:24.650651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ea6c-b7f0-4110-ab4f-133038a1818d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:24.684894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:24.685591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:24.685863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:24.686129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] PATH_INFO: `/v3/users/271a314c0b184c3b9b9c7bcede4bb686` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:24.687316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] RBAC: Authorizing `identity:get_user(user_id=271a314c0b184c3b9b9c7bcede4bb686)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:24.688822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:24.690402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 105/202] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:24 2022] GET /identity/v3/users/271a314c0b184c3b9b9c7bcede4bb686 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:24.696132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:24.732183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:24.800920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a64fd2-1ea4-478f-bce7-33072f4ad614 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:24.834526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:24.835140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:24.835440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:24.835712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:24.905623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:24.907022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:24.942638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:25.008900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 98/203] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 13:14:25.017458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:25.116288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0542b3-8424-4f56-a39d-ac86ac6aead6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:25.150594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:25.151241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:25.151508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:25.151775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:25.152913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:25.154389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79a9d9-aef7-41bb-b6f2-3148f2ec792e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:25.156071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 106/204] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:25 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:25.162218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:25.267511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c982-dbe7-45da-b894-9d57a92d22f3 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:25.302789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:25.303405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:25.303681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:25.303947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:25.304799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] RBAC: Adding query filter params (role.id=d121dc77eff143a186cdd79b2ce341b4, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff, user.id=271a314c0b184c3b9b9c7bcede4bb686) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:25.305109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:25.306552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12a942-af9b-49a7-af2e-bb4c98711830 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:25.309687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:25.313289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 99/205] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:25 2022] GET /identity/v3/role_assignments?user.id=271a314c0b184c3b9b9c7bcede4bb686&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:26.140841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74243-a0a0-4675-a52d-58edf1d0c246 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.141392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74243-a0a0-4675-a52d-58edf1d0c246 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.141392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c74243-a0a0-4675-a52d-58edf1d0c246 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.142741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 107/206] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:26.149399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be32f2-4099-400c-bb4b-3ad65f66b52a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.149937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be32f2-4099-400c-bb4b-3ad65f66b52a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.149937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be32f2-4099-400c-bb4b-3ad65f66b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.170404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-92be32f2-4099-400c-bb4b-3ad65f66b52a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:26.171621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-92be32f2-4099-400c-bb4b-3ad65f66b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:26.194551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bfb3-91ae-442b-89a2-176d1342610f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.194967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bfb3-91ae-442b-89a2-176d1342610f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.195118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bfb3-91ae-442b-89a2-176d1342610f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.196642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 108/207] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:26.201548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.201839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.202087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.219818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:26.221022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:26.223015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:26.247108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 100/208] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:26.260989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc57a-9c17-4006-ad78-c72ab50adb8d None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.261276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc57a-9c17-4006-ad78-c72ab50adb8d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.261553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc57a-9c17-4006-ad78-c72ab50adb8d None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.262890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 101/209] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:26.267892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.268214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.268490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.287333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:26.288571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:26.290821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 109/210] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:26.355344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 102/211] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:26.507764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.549542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.613935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83853b-1443-4a3e-bdfb-6b3d7689bc6e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.614914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:26.648709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.649337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.649608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.649870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.656411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:26.656585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d31e24-daab-4d4d-b900-1b9875a8c89c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.660372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:26.665364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:26.666248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 110/212] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:26.667309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.693996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.694640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.694947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.695218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.705523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:26.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:26.711436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 103/213] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:26.712872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.764362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20c700-e40a-4f60-97b4-cdc7446a0d49 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.798617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.799278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.799585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.799875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.801035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:26.802493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:26.804061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 111/214] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:26.809071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.813327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251cb99-0e8f-471c-a8fb-f95b2bc6b2cc None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.832589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:26.846308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.846915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.847191 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.847457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.848148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:26.848448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:26.849812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:26.857841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 104/215] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:26.858599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.875424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:26.911202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8354172-482b-4574-a6ad-8386d7f84c45 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.943727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.944381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.944661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.944972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.945692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:26.946016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:26.947581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:26.955536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b60c-3cfc-43f5-9073-ed63797ce8be None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:26.955679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 112/216] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:26.956379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:26.993035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:26.993653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:26.993929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:26.994201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:26.994851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:26.996314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.017282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 105/217] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:26 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:27.018162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:27.065020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:27.082938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e9981-5f9e-415e-aa56-e037e4ca1e1e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.113969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:27.122886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.123542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.123822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.124096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] PATH_INFO: `/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.136919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf687a4-4038-43ab-96ba-a3221db288af None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.138517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] RBAC: Authorizing `identity:get_user(user_id=8359bfd2fe514f0d8b2da7d9402b5a2c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.140008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.141779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 113/218] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:26 2022] GET /identity/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:27.147987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:27.176964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.178272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.178787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.179394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] PATH_INFO: `/v3/users/020fb045bc6b4c73a8d725a0c3565cb3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.180640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] RBAC: Authorizing `identity:get_user(user_id=020fb045bc6b4c73a8d725a0c3565cb3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.182135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.188521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 106/219] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:27 2022] GET /identity/v3/users/020fb045bc6b4c73a8d725a0c3565cb3 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:27.189819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:27.256155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d26ac-25b1-4bb2-a23b-5ce23fac196a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.293553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.294246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.294631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.294990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.295733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] RBAC: Adding query filter params (role.id=d121dc77eff143a186cdd79b2ce341b4, user.id=8359bfd2fe514f0d8b2da7d9402b5a2c, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:27.296039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.297770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.305487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 114/220] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 11 13:14:27 2022] GET /identity/v3/role_assignments?user.id=8359bfd2fe514f0d8b2da7d9402b5a2c&scope.system=all&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:27.316662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618aa5b-c828-4c17-8db4-5b47fae5f37e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.325514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:27.357908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.359327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.359327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.359327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.434500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.435965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.541095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 107/221] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 13:14:27.546462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:27.602979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:27.657130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b69b-4ea1-4e1c-9e91-0212e4afaff0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.692869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.693560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.693838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.694109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.695280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.696788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8d150-a0c0-4bcb-8442-55d9cbdc1a05 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.698691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 115/222] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:27 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:27.704584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:27.713136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:27.813125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e700f-7d45-41c3-8ada-e572c03c15fa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:27.847588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:27.848249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:27.848528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:27.848798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:27.849540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] RBAC: Adding query filter params (role.id=0d1fc9612ff54371b3b4106b28cacff2, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff, user.id=020fb045bc6b4c73a8d725a0c3565cb3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:27.849847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:27.851408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf495734-b474-490b-8f3b-2dd1eece8733 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:27.857341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 108/223] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:14:27 2022] GET /identity/v3/role_assignments?user.id=020fb045bc6b4c73a8d725a0c3565cb3&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=0d1fc9612ff54371b3b4106b28cacff2 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:29.308653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff0933-5563-4dc3-bb66-82cb86f60333 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.309236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff0933-5563-4dc3-bb66-82cb86f60333 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.309236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff0933-5563-4dc3-bb66-82cb86f60333 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.310514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 116/224] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:29.316476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefd46-2fd8-4be4-abf1-f6a0eba89ff4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.316990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefd46-2fd8-4be4-abf1-f6a0eba89ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.316990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefd46-2fd8-4be4-abf1-f6a0eba89ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.335748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0dfefd46-2fd8-4be4-abf1-f6a0eba89ff4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:29.336945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0dfefd46-2fd8-4be4-abf1-f6a0eba89ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:29.406943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:29.409662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9780-2998-466a-9205-c52db962f693 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.410043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9780-2998-466a-9205-c52db962f693 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.410321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d9780-2998-466a-9205-c52db962f693 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.411924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 117/225] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:29.417326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.417586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.417705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 109/226] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:29.418236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.436535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:29.437728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:29.465331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724a69-8b5b-4512-b0f6-406646490284 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.465614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724a69-8b5b-4512-b0f6-406646490284 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.465888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724a69-8b5b-4512-b0f6-406646490284 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.467938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 110/227] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:29.472823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.473103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.473373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.493526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:29.495199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:29.512288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 118/228] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:29.565851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 111/229] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:29.671780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:29.723847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:29.775682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96dcce-14c8-4e9e-b3bd-6bf52947daca None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:29.781926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:29.810366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:29.811003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.811276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.811538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.821915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:29.826674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:29.827864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 119/230] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:29.829235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:29.885195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d6c1d-6981-473b-b782-294b79dcd874 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:29.913005 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:29.919704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:29.920369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.920647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.920917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.922013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:29.923471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:29.925071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 112/231] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:29.925829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:29.931366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d0403-035a-42de-b83d-f36a538eb488 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:29.940721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:29.964815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:29.965451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:29.965727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:29.965991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:29.976507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:29.981283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:29.982161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:29.982943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 120/232] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:29.983587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.028224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4d929-d013-439d-b24a-d3823c0c546c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.060514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.061321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.061588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.061857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.062518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:30.062812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.064206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.071993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 113/233] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:30.072738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.082519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb415963-a9b8-4631-bcf0-c3967431565e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.116302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.116940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.117219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.117489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] PATH_INFO: `/v3/groups` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.118116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.119564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.127752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:30.135985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 121/234] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 11 13:14:29 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:30.137024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.170027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:30.182303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813571d-c34c-421a-8fbf-97c08455355e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.220698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.221367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.221648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.221920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.222614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:30.222977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.224705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.234413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 114/235] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:30.240024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.266893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e55f7-c822-4583-9390-c68be8ac821c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.302592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.303237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.303517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.303777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] PATH_INFO: `/v3/users/1039137d8cd14d59a62a34ebc96887d6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.319438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] RBAC: Authorizing `identity:get_user(user_id=1039137d8cd14d59a62a34ebc96887d6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.321089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.323020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 122/236] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:30 2022] GET /identity/v3/users/1039137d8cd14d59a62a34ebc96887d6 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:30.328544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.357339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3311339-0974-42b7-a6db-72382c176422 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.381047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:30.393388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.394164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.394529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.394906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] PATH_INFO: `/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.396145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] RBAC: Authorizing `identity:get_user(user_id=8359bfd2fe514f0d8b2da7d9402b5a2c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.398095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.404352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 115/237] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:30 2022] GET /identity/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:30.405290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:30.421506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:30.436607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74fef-c5fb-4d14-8c1a-3c4b038d84ef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.470771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.471385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.471662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.471930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.472689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] RBAC: Adding query filter params (role.id=0d1fc9612ff54371b3b4106b28cacff2, user.id=1039137d8cd14d59a62a34ebc96887d6, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:30.473005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.474448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd391d9-0306-43a1-abf5-fc239c9527b5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.479433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 123/238] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 11 13:14:30 2022] GET /identity/v3/role_assignments?user.id=1039137d8cd14d59a62a34ebc96887d6&scope.system=all&role.id=0d1fc9612ff54371b3b4106b28cacff2 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:30.523944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7531f4d-87f0-429e-9e21-70990c8c5e54 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:30.560467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:30.561143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:30.561466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:30.561798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] PATH_INFO: `/v3/system/users/8359bfd2fe514f0d8b2da7d9402b5a2c/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:30.563285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8359bfd2fe514f0d8b2da7d9402b5a2c, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:30.564763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aad34-486c-4075-b8e3-b7071cbbebb0 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:30.579537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 116/239] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 11 13:14:30 2022] PUT /identity/v3/system/users/8359bfd2fe514f0d8b2da7d9402b5a2c/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:32.280887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d988b-f1ed-41ab-b2f1-5f20176944be None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.281446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d988b-f1ed-41ab-b2f1-5f20176944be None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.281446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d988b-f1ed-41ab-b2f1-5f20176944be None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.282815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 124/240] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:32.288153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d4c9-9160-4cbe-a0c3-6af607aa1ecb None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.288712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d4c9-9160-4cbe-a0c3-6af607aa1ecb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.288789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d4c9-9160-4cbe-a0c3-6af607aa1ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.307264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2216d4c9-9160-4cbe-a0c3-6af607aa1ecb None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:32.308482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2216d4c9-9160-4cbe-a0c3-6af607aa1ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:32.360590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:32.380185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 117/241] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:32.574814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af8ff-3d05-4b5d-9339-79ffc6ab4aff None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.575051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af8ff-3d05-4b5d-9339-79ffc6ab4aff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.575491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af8ff-3d05-4b5d-9339-79ffc6ab4aff None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.580153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 125/242] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:32.582308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6e2b0-c071-44bd-a0ab-573d59b0bf01 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.582554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6e2b0-c071-44bd-a0ab-573d59b0bf01 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.582846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6e2b0-c071-44bd-a0ab-573d59b0bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.603400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bbf6e2b0-c071-44bd-a0ab-573d59b0bf01 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:32.604652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bbf6e2b0-c071-44bd-a0ab-573d59b0bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:32.645834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35af8ff-3d05-4b5d-9339-79ffc6ab4aff None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:32.676065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 118/243] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:32.716294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582215c1-6932-479d-b606-84e43854d96b None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.716822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582215c1-6932-479d-b606-84e43854d96b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.717093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582215c1-6932-479d-b606-84e43854d96b None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.719021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 119/244] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:32.724227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.724504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.724768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.742382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:32.743550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:32.752915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35af8ff-3d05-4b5d-9339-79ffc6ab4aff None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:32.791760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:32.792781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.793435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.793857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.795055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:32.797334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:32.799526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 126/245] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:32.805113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:32.810151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 120/246] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:32.827363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:32.909400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201849b7-e28c-4a40-9462-06013903bee9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:32.938425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:32.943212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:32.943831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:32.944103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:32.944399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] PATH_INFO: `/v3/groups` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:32.945100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:32.946548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:32.963281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 127/247] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 11 13:14:32 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:14:32.989216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.051374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270633-15d3-4b86-9afb-e802635c3013 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.072757 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.088901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.089900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.090349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.090618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.102718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:33.107750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:33.108707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:33.109484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:33.113904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 121/248] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:33.114890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.164644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.191160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138533b3-985a-44df-a24f-796585e33d31 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.234072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.234990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.235270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.235536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.242434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d994071-438d-4128-9203-537658d8bb1b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.245304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:33.259938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:33.261119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:33.261926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 128/249] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:33.274772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.295612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.296426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.296772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.297119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.297866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:33.298177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.299561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.310058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 122/250] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:33.316374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.387781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999762d-4ddc-41d8-aaeb-7014ba5913b5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.437037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.438535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.438986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.439425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.440730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:33.441452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.443179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.443283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1e984-ea9a-41df-ae34-6896c15b6ed1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.453023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.460465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 129/251] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:33.469188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.491801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.492926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.493210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.493487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] PATH_INFO: `/v3/users/1039137d8cd14d59a62a34ebc96887d6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.494660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] RBAC: Authorizing `identity:get_user(user_id=1039137d8cd14d59a62a34ebc96887d6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.496804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.504660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 123/252] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/users/1039137d8cd14d59a62a34ebc96887d6 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:33.505536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.550234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.599697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672bd03-66b1-45a0-a751-cd3f90c0a0bf None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.640216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.640804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.641078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.641345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] PATH_INFO: `/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.642519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] RBAC: Authorizing `identity:get_user(user_id=8359bfd2fe514f0d8b2da7d9402b5a2c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.644024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.650377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78416-c786-47cd-9d5f-7986349342c4 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.650860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 130/253] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/users/8359bfd2fe514f0d8b2da7d9402b5a2c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:33.652691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:33.697217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.697884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.698159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.698428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] PATH_INFO: `/v3/system/users/1039137d8cd14d59a62a34ebc96887d6/roles/0d1fc9612ff54371b3b4106b28cacff2` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.700339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1039137d8cd14d59a62a34ebc96887d6, role_id=0d1fc9612ff54371b3b4106b28cacff2)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.701722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e0510-e6f3-408d-946e-b482efbc85ed None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.712571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 124/254] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 11 13:14:33 2022] PUT /identity/v3/system/users/1039137d8cd14d59a62a34ebc96887d6/roles/0d1fc9612ff54371b3b4106b28cacff2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:33.745325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:33.789198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b1f0a-c06a-4c61-9d6f-cc35292f507a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:33.831134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:33.832244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:33.832776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:33.833234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:33.834182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] RBAC: Adding query filter params (role.id=d121dc77eff143a186cdd79b2ce341b4, user.id=8359bfd2fe514f0d8b2da7d9402b5a2c, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:33.834694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:33.836631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b8e0e-78de-46b4-b833-e9c8a5d00f08 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:33.851534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 131/255] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 11 13:14:33 2022] GET /identity/v3/role_assignments?user.id=8359bfd2fe514f0d8b2da7d9402b5a2c&scope.system=all&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:35.498989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a70d53-f20d-467f-ab77-14c7a649858a None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:35.499498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a70d53-f20d-467f-ab77-14c7a649858a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:35.499572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a70d53-f20d-467f-ab77-14c7a649858a None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:35.501169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 125/256] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:35.506353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b3a1a-5ea0-4c58-9559-63e9d34fa2d4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:35.506756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b3a1a-5ea0-4c58-9559-63e9d34fa2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:35.506927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b3a1a-5ea0-4c58-9559-63e9d34fa2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:35.533134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7a8b3a1a-5ea0-4c58-9559-63e9d34fa2d4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:35.534333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7a8b3a1a-5ea0-4c58-9559-63e9d34fa2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:35.618956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 132/257] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:14:35.720811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10558ea-6f0d-4356-815c-f32b5ef0a879 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:35.721128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10558ea-6f0d-4356-815c-f32b5ef0a879 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:35.721352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10558ea-6f0d-4356-815c-f32b5ef0a879 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:35.723000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 126/258] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:35.728210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:35.728494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:35.728761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:35.757860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:35.758952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:35.771358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 133/259] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:14:35.978114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10558ea-6f0d-4356-815c-f32b5ef0a879 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:35.993658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10558ea-6f0d-4356-815c-f32b5ef0a879 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:35.998040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:35.998677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:35.998951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:35.999217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.013596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:36.018335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:36.019179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:36.021399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 127/260] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:36.029252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.042596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71990f-7500-4bb8-829f-c90d9015e587 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.052213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.052842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.053114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.053382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.054135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:36.054441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.056278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.056278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:14:36.056278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:14:36.057188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.072170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 134/261] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:36.078358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.092599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130815a-f2e1-432c-b989-5c471ae5260e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.100263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.101024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.101260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.101531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.113296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.115101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.115101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:14:36.115101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:14:36.116341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.118059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1201 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:36.123804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.135404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56e693-20b3-47c4-894c-aad0d36b2f06 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.140023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.140776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.141051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.141324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.142480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.144753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.146616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 135/263] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:36.153312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.164495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce60e4-aa52-400f-a59a-c35e98174630 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.169861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.170593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.170869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.171139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.173245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, group_id=a4399e082ddc468a9a0309bc67ba05d0, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.174267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.174267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:14:36.174267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:14:36.176363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.201511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 129/264] 10.222.0.15 () {58 vars in 1463 bytes} [Tue Oct 11 13:14:36 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:14:36.246930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.258490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aff98-0e27-41d9-8092-16e3fae6e769 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.319129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:36.325471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.326134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.326409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.326677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.344348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:36.350033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:36.352215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:36.354249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 136/265] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 105 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:36.357150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.369604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139c25-1c94-410b-9256-b20b90e7e374 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.375649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.376407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.376686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.376955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.377701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:36.378005 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.379152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.379152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:14:36.379152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:14:36.380051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.391109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 130/266] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:36.397396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.410224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae78a-0514-49e4-9ee3-1ec9b3af17f1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.415741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.416523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.416804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.417072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.418235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.419326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.419326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:14:36.419326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:14:36.420634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.422405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 137/267] 10.222.0.15 () {56 vars in 1201 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:36.428765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.439870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f962b6f-6fff-4f4a-bcb6-38aac7c8ac48 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.448106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.448856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.449136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.449456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.450736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.452959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a028cc0-888e-421f-9021-c2c785f1e4dd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.455136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 131/268] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:14:36 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:36.460813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:36.471574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02490be7-8fdd-4d9a-a25e-f7e0d87039a5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:36.476231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:36.476915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.477240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.477510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.479748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbdcf3a75044d9db0deed607efeb8b1, group_id=a4399e082ddc468a9a0309bc67ba05d0, role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:36.481635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:14:36.481635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:14:36.481635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:14:36.482888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758acac5-e831-4070-9077-d43e56dd1849 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:36.500882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 138/269] 10.222.0.15 () {58 vars in 1463 bytes} [Tue Oct 11 13:14:36 2022] PUT /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/314f2033c1db4762926d37c1532eb2f1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:36.829508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce388c-169e-4d03-aad9-a3b2726d7158 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.830235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce388c-169e-4d03-aad9-a3b2726d7158 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.830235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce388c-169e-4d03-aad9-a3b2726d7158 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.831713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 132/270] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:36.838316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e1cf-f7ae-46af-ae49-ba1f207f3532 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.838898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e1cf-f7ae-46af-ae49-ba1f207f3532 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.838898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e1cf-f7ae-46af-ae49-ba1f207f3532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.869200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-aed6e1cf-f7ae-46af-ae49-ba1f207f3532 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:36.870321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aed6e1cf-f7ae-46af-ae49-ba1f207f3532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:36.954387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 139/271] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:36.964727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2f0b-d59f-4c32-9313-f95bbf70dbb7 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.964727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2f0b-d59f-4c32-9313-f95bbf70dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.964727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2f0b-d59f-4c32-9313-f95bbf70dbb7 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.964727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 133/272] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:36.969913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:36.970198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:36.970472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:36.993188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:36.994473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:37.074678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 140/273] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:14:37.295772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2f0b-d59f-4c32-9313-f95bbf70dbb7 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.363517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:37.424570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2f0b-d59f-4c32-9313-f95bbf70dbb7 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.475561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.476361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.476635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.476904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.490993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:37.496375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:37.497237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:37.498037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 134/274] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:37.502487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.511141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.515769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434cc08-942c-436f-9730-532a303f6c94 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.527712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.528689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.529166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.529655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.553848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:37.559456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:37.560392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:37.561184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:37.568145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 141/275] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:37.575393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.600375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a4d7d-8e40-45e3-9107-2a435e3796c3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.609609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.610570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.611069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.611541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.612547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:37.613069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:37.614946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:37.629090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 142/276] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:37.655865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.670140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3091c-fde0-410f-8efb-cde8cffd7d24 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.682182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.683167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.683664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.684147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.684567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28651-b33c-4680-8271-57e7891850a8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.688341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:37.690537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:37.692415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 143/277] 10.222.0.15 () {56 vars in 1201 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:37.703119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:37.721798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.739699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02f9d0-ffd6-4575-9a90-04ee4842b50f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.744636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.745324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.745609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.745887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.746813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:37.747117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:37.748671 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:37.757553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.758501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.758977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.759465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.763449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:37.765771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:37.767370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 135/278] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:37.768157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 144/279] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:37.774369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.774544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:37.784894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee04d2-0c52-4a22-9fca-1ee7c0320a47 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.798781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.799883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.800382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.800870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.805526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, group_id=a4399e082ddc468a9a0309bc67ba05d0, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:37.808599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79e29-9c6b-4fc1-abbf-d8721f3960a9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:37.830558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 145/280] 10.222.0.15 () {58 vars in 1463 bytes} [Tue Oct 11 13:14:37 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:37.852157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0a26-bc83-4889-a18e-2b4ad2e464a5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.853238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0a26-bc83-4889-a18e-2b4ad2e464a5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.853710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0a26-bc83-4889-a18e-2b4ad2e464a5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.855469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 146/281] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:37.862105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.862724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:37.863220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:37.897744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:37.899378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:37.918168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:37.931595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c3dcd-62b2-4190-8f35-6fd5942cebc7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:37.998160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:37.999188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:37.999655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.000111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] PATH_INFO: `/v3/users/1039137d8cd14d59a62a34ebc96887d6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.004892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] RBAC: Authorizing `identity:get_user(user_id=1039137d8cd14d59a62a34ebc96887d6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.006657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7c8d5-d89b-4c1b-abbc-582481c8dd31 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.006899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 147/282] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 13:14:38.009089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 136/283] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:14:37 2022] GET /identity/v3/users/1039137d8cd14d59a62a34ebc96887d6 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.014579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.169738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1693eee-3b60-4601-8312-a30cdf6a79c4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.215543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.216181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.216465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.216733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.217466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] RBAC: Adding query filter params (role.id=0d1fc9612ff54371b3b4106b28cacff2, user.id=1039137d8cd14d59a62a34ebc96887d6, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:38.217801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.219255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bb29c-045b-4c10-81a7-ba88efd8e61e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.232069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 148/284] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/role_assignments?user.id=1039137d8cd14d59a62a34ebc96887d6&scope.system=all&role.id=0d1fc9612ff54371b3b4106b28cacff2 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.296941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26763079-4745-4e29-9ae1-95744eccdd11 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.297589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26763079-4745-4e29-9ae1-95744eccdd11 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.298158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26763079-4745-4e29-9ae1-95744eccdd11 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.299809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 137/285] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:38.313865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.314166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.314414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.341097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:38.342310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:38.353820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 149/286] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 13:14:38.361385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26763079-4745-4e29-9ae1-95744eccdd11 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.376047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26763079-4745-4e29-9ae1-95744eccdd11 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.382126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.383247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.383715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.384280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.397322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:38.403467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:38.404405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:38.405216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:38.410148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 138/287] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:38.414369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.424905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66186b1-abbc-4c3c-935f-dbe2d0861661 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.429893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.430946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.431416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.432184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.433161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:38.433650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.438673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:38.452196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 150/288] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.453912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.464874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5f04-42f9-4090-a061-8645457e3705 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.469946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.470853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.471322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.471776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.473231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.475506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.478849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 139/289] 10.222.0.15 () {56 vars in 1201 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.483928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.494957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b595cd-b076-4449-a73b-fdc9585df9b4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.499841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.500769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.501272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.501727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.503157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.505629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.510402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 151/290] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.514963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.526009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b79e4-be54-4c8a-9f61-7eed16408022 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.532312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.534038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.534544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.535067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.537660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff, group_id=a4399e082ddc468a9a0309bc67ba05d0, role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.540756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.560159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 140/291] 10.222.0.15 () {58 vars in 1463 bytes} [Tue Oct 11 13:14:38 2022] PUT /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff/groups/a4399e082ddc468a9a0309bc67ba05d0/roles/314f2033c1db4762926d37c1532eb2f1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:38.735765 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.747026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87efef8-d63c-443a-9c87-c14e7a17db7c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.818277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.819304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.819878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.820409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.832877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:38.840015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:38.840885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:38.841665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:38.846145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 152/292] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:38.849704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.860392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640694da-d9cb-4b65-8ac5-4d2f5441579e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.866379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.867263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.867730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.868265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.869167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:38.869672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.871573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.886730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 141/293] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.888420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.899608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c4fd8-64aa-44f5-a0aa-aab7f9af51d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.906493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.907424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.907891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.908468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] PATH_INFO: `/v3/groups/0e2c1d3322614a4aa835ca5d2f79750a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.916477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] RBAC: Authorizing `identity:get_group(group_id=0e2c1d3322614a4aa835ca5d2f79750a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.918800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.920741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 153/294] 10.222.0.15 () {56 vars in 1201 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/groups/0e2c1d3322614a4aa835ca5d2f79750a => generated 229 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.926607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.933713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:38.939324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bead1-74d3-42f6-ac1d-02a3c1e082dd admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.946847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.947561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.948247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.948850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] PATH_INFO: `/v3/projects/81928fdc604942f18e8f5f0944263250` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.950750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] RBAC: Authorizing `identity:get_project(project_id=81928fdc604942f18e8f5f0944263250)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.953049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d7ab-c100-42bb-a455-1faf5331d063 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:38.955341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 142/295] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 11 13:14:38 2022] GET /identity/v3/projects/81928fdc604942f18e8f5f0944263250 => generated 344 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:38.961249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:38.973793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9df79-9255-4d1a-9083-8df960872ad2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:38.981774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:38.982449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:38.982719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:38.982984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] PATH_INFO: `/v3/projects/81928fdc604942f18e8f5f0944263250/groups/0e2c1d3322614a4aa835ca5d2f79750a/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:38.984889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81928fdc604942f18e8f5f0944263250, group_id=0e2c1d3322614a4aa835ca5d2f79750a, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:38.987567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf25ff3-fdd5-4a54-ba59-466acb8f15e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:39.015353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 154/296] 10.222.0.15 () {58 vars in 1463 bytes} [Tue Oct 11 13:14:38 2022] PUT /identity/v3/projects/81928fdc604942f18e8f5f0944263250/groups/0e2c1d3322614a4aa835ca5d2f79750a/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:39.706362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649fae8-2a8e-47ce-b2ea-d30d7bb4b9b2 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:39.706767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649fae8-2a8e-47ce-b2ea-d30d7bb4b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:39.706892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649fae8-2a8e-47ce-b2ea-d30d7bb4b9b2 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:39.708678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 143/297] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:39.715796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:39.716109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:39.716379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:39.750020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:39.751222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:39.830243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e87f25-b566-4a15-9fd4-65955fc9a3f6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:39.830873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e87f25-b566-4a15-9fd4-65955fc9a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:39.831367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e87f25-b566-4a15-9fd4-65955fc9a3f6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:39.835815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 144/298] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:39.837555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 155/299] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:39.842939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:39.843454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:39.843923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:39.876545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:39.877725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:39.962394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 145/300] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:40.212323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.266312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:40.295868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.349182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f68af-ff95-4a10-b014-b9771294b573 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.410371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.411029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.411303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.411596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:40.416703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344be2-b35c-4245-819c-feb89934ffa8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.429903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:40.434758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:40.435609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:40.436447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 156/301] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:40.444159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.458735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.459728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.460344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.460805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:40.463887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:40.476437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:40.481472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:40.482367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:40.483152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:40.486834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 146/302] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:40.492396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.569250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ef00-ef26-4b31-af08-0394c77fbccb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.577423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:40.633372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.634360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.634828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.635311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:40.636194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:40.636702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:40.638274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:40.655698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 157/303] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:40.660879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.662222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659be5c-4d7c-44e4-9dda-550c78ff7fd6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.705035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.705655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.705928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.706194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:40.707536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:40.707852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:40.709465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:40.729333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 147/304] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:40.734934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:40.828248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:40.830172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7eb82e-038f-47d3-a16e-09dfdd9ef8ea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.895896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.896583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.897174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.897447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:40.905358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20c5c-8bd0-4f5b-b120-f74689012e33 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:40.936486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:40.978998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:40.980579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead9d45-5d36-4abc-80a3-773b1e6a8998 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:40.996673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:40.997305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:40.997580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:40.997855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.103054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.105389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768de00-d793-4c77-9ab9-3d312bf81b8d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.135821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 158/305] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 13:14:41.137511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d652d-1fb0-4674-ba04-6bb14f05d6df None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.137688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d652d-1fb0-4674-ba04-6bb14f05d6df None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.137947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d652d-1fb0-4674-ba04-6bb14f05d6df None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.143979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 159/306] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:41.143979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d652d-1fb0-4674-ba04-6bb14f05d6df None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.177325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.244069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 148/307] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 13:14:41.246397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.246952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.247416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.252148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.269989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:41.271494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:41.306894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d652d-1fb0-4674-ba04-6bb14f05d6df None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.349080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.349727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.350002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.350272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.353005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.353792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 149/308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:41.356557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.356557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 160/309] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:41.360139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.361970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.496219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.522556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc95d-0011-439a-97c6-ac40288319e6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.553510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0064cd-0b1f-42d1-9128-68ab66abd489 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.587124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.588062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.588587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.589025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.592756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.594422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651abd-7ec5-44a5-8d1b-6e00babb7e36 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.596732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 150/310] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:41.598222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad653-28a3-4608-bce3-b7a307022191 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.598696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad653-28a3-4608-bce3-b7a307022191 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.599151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad653-28a3-4608-bce3-b7a307022191 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.602237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 151/311] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:41.603778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad653-28a3-4608-bce3-b7a307022191 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.610524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.611237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.611516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.611790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] PATH_INFO: `/v3/projects/efbdcf3a75044d9db0deed607efeb8b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.613029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=efbdcf3a75044d9db0deed607efeb8b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.614701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92a851-ad45-40ea-9e39-314ea438d8f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.618218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 161/312] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/projects/efbdcf3a75044d9db0deed607efeb8b1 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:41.618218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.618218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.618218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.625562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.639792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:41.640999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:41.714975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 162/313] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:41.715846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.725632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad653-28a3-4608-bce3-b7a307022191 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.770768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9264c-0923-4184-981c-973811e896df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.771703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9264c-0923-4184-981c-973811e896df None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.772702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9264c-0923-4184-981c-973811e896df None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.773183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9264c-0923-4184-981c-973811e896df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.774497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9264c-0923-4184-981c-973811e896df None admin] RBAC: Adding query filter params (group.id=a4399e082ddc468a9a0309bc67ba05d0, role.id=314f2033c1db4762926d37c1532eb2f1, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:41.775083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9264c-0923-4184-981c-973811e896df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.776964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9264c-0923-4184-981c-973811e896df None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.783153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 152/314] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/role_assignments?group.id=a4399e082ddc468a9a0309bc67ba05d0&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=314f2033c1db4762926d37c1532eb2f1 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:41.784410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9264c-0923-4184-981c-973811e896df None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.803172 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.840011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4181f5-dbd4-4b3e-9b38-d5559ff735ee None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.873625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:41.881073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.882211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.882695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.883186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.884525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:41.886242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:41.888090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 163/315] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:41.897580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:41.926681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9264c-0923-4184-981c-973811e896df None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:41.971920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:41.972594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:41.972875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:41.973142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:41.986657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:41.992338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:41.993277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:41.994489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 153/316] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:41.999668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.093759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.094978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242959b-4031-4656-8cc2-ecf1df694de0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.131825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1a223-fc3a-4732-a0d2-c0676152f990 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.141308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.141933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.142208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.142839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.143580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] RBAC: Adding query filter params (group.id=a4399e082ddc468a9a0309bc67ba05d0, role.id=d121dc77eff143a186cdd79b2ce341b4, scope.project.id=efbdcf3a75044d9db0deed607efeb8b1) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:42.144228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:42.145662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b1bcd-872b-4c58-932c-069a85dc091a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:42.152728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 164/317] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/role_assignments?group.id=a4399e082ddc468a9a0309bc67ba05d0&scope.project.id=efbdcf3a75044d9db0deed607efeb8b1&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:42.153941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0768f-dcbc-4f78-a2a8-ea591d77c363 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.154213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0768f-dcbc-4f78-a2a8-ea591d77c363 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.154485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0768f-dcbc-4f78-a2a8-ea591d77c363 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.157204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 165/318] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:42.158172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0768f-dcbc-4f78-a2a8-ea591d77c363 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.172954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.173588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.173866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.174193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.174802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] RBAC: Adding query filter params (name=member) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:42.175199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:42.176537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2edb34-38a5-4909-854f-469cbbbaf16c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:42.192146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 154/319] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:42.192146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.192146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.192146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.214975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:42.215870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:42.248266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.300723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0768f-dcbc-4f78-a2a8-ea591d77c363 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.309651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 155/320] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:42.311595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.357545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.358554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.359058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.359514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.373360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:42.380159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:42.381041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:42.381818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:42.388498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 166/321] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:42.390668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.398730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.448047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b43fc-9788-406b-9d0b-b72068590e88 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.488557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.490041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.490332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.490607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.501926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc55d7a-9cee-4a76-b294-8030cd19c413 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.529446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.542361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.542979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.543275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.543521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.544226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:42.544528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:42.545900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:42.554532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 167/322] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:42.560982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.567518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:42.570779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:42.650226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.684444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9cefb-6a8d-4c9d-bf00-41776404a59d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.688598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 156/323] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:42.689114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.719330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.719945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.720244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.720514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.777476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.791489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:42.793027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:42.804410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45559646-9584-4ebe-82fb-2777c325d8b5 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:42.842779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:42.843982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:42.843982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:42.843982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:42.857115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:42.861944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:42.862917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:42.863719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 157/324] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:42.864482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.895988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:42.904105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 168/325] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 13:14:42.904957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:42.994923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86c064-7fcf-4b9d-813e-f70ef2c49a69 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.000716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.013291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad072bea-cc2d-40a7-b30a-fa24dd42e69c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.034363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.035041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.035273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.035540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.037112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.039319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.041347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 158/326] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.041480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.057262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.057883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.058160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.058426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.059106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:43.059407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.060826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.071552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 169/327] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.072518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.153258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.157428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf385-8e5d-4b77-9380-fc4264a79fcc None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.182257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72a45-8827-4cab-b280-9ee25468a3f0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.197992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.198961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.199444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.199896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] PATH_INFO: `/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.201044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] RBAC: Authorizing `identity:get_project(project_id=bcefafc8b92b4d90a4e12fc46f21b3ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.202485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.204156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 159/328] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/projects/bcefafc8b92b4d90a4e12fc46f21b3ff => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.208482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.218324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.218988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.219265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.219540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.220660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.222096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.223679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 170/329] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.224459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.241785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.307480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e49fd5-faca-4571-b84b-5041845e3010 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.324023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58210e3-b99d-4acc-8999-6ecbfee3bba1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.344994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.345615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.345888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.346162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.357876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.358509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.358782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.359059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] PATH_INFO: `/v3/groups/a4399e082ddc468a9a0309bc67ba05d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.360188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] RBAC: Authorizing `identity:get_group(group_id=a4399e082ddc468a9a0309bc67ba05d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.361646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.363213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 171/330] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/groups/a4399e082ddc468a9a0309bc67ba05d0 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.363965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.377140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.415664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.417092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.447816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.463905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8410492-8285-4003-9a79-bd08020428bc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.497612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.498208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.498477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.498739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.499467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] RBAC: Adding query filter params (group.id=a4399e082ddc468a9a0309bc67ba05d0, role.id=d121dc77eff143a186cdd79b2ce341b4, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:43.499769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.501222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.512606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 172/331] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/role_assignments?group.id=a4399e082ddc468a9a0309bc67ba05d0&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=d121dc77eff143a186cdd79b2ce341b4 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.512606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.518547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 160/332] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:14:43.524722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.592087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.637015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c32f9-d9b0-4663-ae5e-daec0e0e06d2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.640176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3370a1-8b3d-42dd-a8d5-cdafbb876c0d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.675182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.675758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.676034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.676149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.676324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] PATH_INFO: `/v3/projects/81928fdc604942f18e8f5f0944263250` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.676678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.676951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.677230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.677504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] RBAC: Authorizing `identity:get_project(project_id=81928fdc604942f18e8f5f0944263250)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.677959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] RBAC: Adding query filter params (group.id=a4399e082ddc468a9a0309bc67ba05d0, role.id=314f2033c1db4762926d37c1532eb2f1, scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:43.678274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.679648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.679775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.681335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 161/333] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/projects/81928fdc604942f18e8f5f0944263250 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:43.686808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.690966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 173/334] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/role_assignments?group.id=a4399e082ddc468a9a0309bc67ba05d0&scope.project.id=bcefafc8b92b4d90a4e12fc46f21b3ff&role.id=314f2033c1db4762926d37c1532eb2f1 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.809240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194019df-8472-4463-8ca0-bcc7fadc5d5d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.840627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.847224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.847882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:43.848195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:43.848482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] PATH_INFO: `/v3/groups/0e2c1d3322614a4aa835ca5d2f79750a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:43.849646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] RBAC: Authorizing `identity:get_group(group_id=0e2c1d3322614a4aa835ca5d2f79750a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:43.851119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e666991-88ab-4e39-9bb9-e968be674a56 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:43.852781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 162/335] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/groups/0e2c1d3322614a4aa835ca5d2f79750a => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:43.858782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:43.874959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:43.965479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d28cc-fcc1-4a9b-969a-5789c9678f1b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:43.999205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:43.999820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:44.000097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:44.000407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:44.001142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] RBAC: Adding query filter params (group.id=0e2c1d3322614a4aa835ca5d2f79750a, role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=81928fdc604942f18e8f5f0944263250) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:44.001447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:44.002902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee432c6-a819-4c79-a324-44be20710202 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:44.008787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 174/336] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 11 13:14:43 2022] GET /identity/v3/role_assignments?group.id=0e2c1d3322614a4aa835ca5d2f79750a&scope.project.id=81928fdc604942f18e8f5f0944263250&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:46.475001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6bf0-9be8-4599-b154-cca484abd697 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:46.475525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6bf0-9be8-4599-b154-cca484abd697 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:46.475525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6bf0-9be8-4599-b154-cca484abd697 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:46.477114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 163/337] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:46.482538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ad7aa-d960-4510-9e1f-f1a92e542453 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:46.482963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ad7aa-d960-4510-9e1f-f1a92e542453 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:46.483141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ad7aa-d960-4510-9e1f-f1a92e542453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:46.505042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-123ad7aa-d960-4510-9e1f-f1a92e542453 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:46.506214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-123ad7aa-d960-4510-9e1f-f1a92e542453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:46.593063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 175/338] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:46.710552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b50f-35e5-46aa-a46f-43b2142d2adb None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:46.710870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b50f-35e5-46aa-a46f-43b2142d2adb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:46.711105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b50f-35e5-46aa-a46f-43b2142d2adb None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:46.712502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 164/339] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:46.717845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:46.718127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:46.718392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:46.736215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:46.742494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:46.743656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:46.826332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 176/340] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:47.024746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36b50f-35e5-46aa-a46f-43b2142d2adb None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.157789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36b50f-35e5-46aa-a46f-43b2142d2adb None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.166464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.196978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.197900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.198177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.198448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.217505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.218966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:47.250029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:47.251229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:47.252299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:47.252299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:47.252299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:47.252299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 165/341] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:47.259627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.358841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240ff71-39c5-408f-92cd-90dd9b05e802 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.367900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:47.405324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.406432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.406715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.406992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.410545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:47.424053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.426211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.427342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978634a-95ed-486f-9527-9e03abc8190e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bfa38ef0-c87f-4434-ae4e-8e99f1513ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:47.447452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:47.448448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:47.449292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:47.449292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:47.449292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:47.452268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 177/342] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:47.453621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150191de-348f-43fd-b1b0-25e2d6b8aa64 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.453898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150191de-348f-43fd-b1b0-25e2d6b8aa64 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.455193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150191de-348f-43fd-b1b0-25e2d6b8aa64 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.456522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 178/343] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:47.457535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b4a7f-fc13-4299-984b-6989c9b9eed5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.457803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b4a7f-fc13-4299-984b-6989c9b9eed5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.458121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b4a7f-fc13-4299-984b-6989c9b9eed5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.460519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 179/344] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:47.460519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b4a7f-fc13-4299-984b-6989c9b9eed5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.473938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.474636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.474908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.475175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.475857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:47.476192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.477553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c0316-31b7-4b47-b1fb-fd7a47b8ed7a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.494189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 166/345] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:47.494189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abfc51-8891-4dbd-9109-689f43fcb6ba None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.494189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abfc51-8891-4dbd-9109-689f43fcb6ba None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.494189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abfc51-8891-4dbd-9109-689f43fcb6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.511613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-44abfc51-8891-4dbd-9109-689f43fcb6ba None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:47.512818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-44abfc51-8891-4dbd-9109-689f43fcb6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:47.574609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b4a7f-fc13-4299-984b-6989c9b9eed5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.584834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 167/346] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:47.586331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866756-b055-4f68-a841-47b39d1676de None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.586748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866756-b055-4f68-a841-47b39d1676de None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.587138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866756-b055-4f68-a841-47b39d1676de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.610615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.611455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.611888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.612329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.613146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:47.613596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.614128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8a866756-b055-4f68-a841-47b39d1676de None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:47.615304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8a866756-b055-4f68-a841-47b39d1676de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:47.615453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.629863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 180/347] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:47.631119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.653587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:47.688377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 168/348] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:47.692984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a866756-b055-4f68-a841-47b39d1676de None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.701535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:47.786123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a3a32-be58-4af5-a950-bf2f789a7223 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.851840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.852499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.852775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.853045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.853714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.855217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.876857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a866756-b055-4f68-a841-47b39d1676de None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:47.904569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 181/349] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:47 2022] POST /identity/v3/users => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:47.905657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:47.946252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:47.947572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.949168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.950287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:47.951659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:47.955845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20871b31-8e34-45d1-8d21-b97a85205be3 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:47.994973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 169/350] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:47 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:47.996763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0d7a-2f82-4b2a-827a-bbdd24bfb6a7 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:47.997256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0d7a-2f82-4b2a-827a-bbdd24bfb6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:47.997722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf0d7a-2f82-4b2a-827a-bbdd24bfb6a7 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.004562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 170/351] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:48.006349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.006800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.007437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.030589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:48.032315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:48.057661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.067592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cbe3b-b61b-4c72-b14f-3558e9ac6f09 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.071804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.121799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.122803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.123290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.123704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.131228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 171/352] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:48.131228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.156660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.158377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:48.180024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:48.181203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:48.182117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:48.182117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:48.182117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.182117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 182/353] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:48.193259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.259371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6331d-894e-4044-82b5-657bacbc1694 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.306044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3cc56-2855-47d9-a1bc-ec0297c3d826 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.313309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.313958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.314239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.314503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.334867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.336455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.345388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.346000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.346321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.346586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.347225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:48.347568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.348987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.361284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 183/354] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:48.362262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:48.363148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:48.364552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:48.364552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:48.364552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:48.364552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 172/355] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:48.368969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.370476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.388586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.391690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.473467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec9110-ee94-425b-acef-f32031e91be0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.510673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.511630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.512075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.512253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc09d1-fad6-44c1-afab-0cef78884926 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.513010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.514097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.516662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.547580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 184/356] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:48 2022] POST /identity/v3/users => generated 248 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:14:48.549423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.554366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.554962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.555372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.555716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.556540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:48.556937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.558550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.577822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 173/357] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:48.584379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.684873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.727214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.730833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e40c4-e504-40d7-a185-d1a380df957f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.770683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.771365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.771650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.772307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.773066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.774464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.785199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 13:14:48.785199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 13:14:48.785199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [parameters: {'id': '8040efdb4cba4ab5b71cf523a0213286', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 13:14:48.785199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 13:14:48.789745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e4c8b-496e-4481-9976-ccc782029686 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 11 13:14:48.808211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return ref.to_dict() Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:14:48.809183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 13:14:48.810590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 13:14:48.811703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:14:48.812615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:14:48.813385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'id': '8040efdb4cba4ab5b71cf523a0213286', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:48.814139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 13:14:48.816825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.818826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 185/358] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 11 13:14:48 2022] POST /identity/v3/roles => generated 144 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:48.818826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:48.842580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.843631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.844185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.844645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.845477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:48.847188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:48.880293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 174/359] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:48 2022] POST /identity/v3/users => generated 249 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:14:48.933464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242250e9-82b8-4b99-8472-0c4a35175961 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:48.972011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:48.972677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:48.972955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:48.973273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:48.976555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:48.987277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a6c984d3-fd44-4683-8c8c-6a15c39aeb2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:48.993012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:48.993967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 13:14:48.994748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:48.997447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 186/360] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:14:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:49.000816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:49.122862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81fea-4124-4ae5-9918-739b36ebd43a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:49.147409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:49.158446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:49.159675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:49.160320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:49.160612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:49.161289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:49.161600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:49.163123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d9399-e0db-47ff-84ee-983c8b0705b1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:49.171471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 175/361] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:50.484257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176570-bd1d-4705-8733-6d7124de4f36 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:50.484806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176570-bd1d-4705-8733-6d7124de4f36 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:50.484806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176570-bd1d-4705-8733-6d7124de4f36 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:50.486462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 187/362] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:50.491728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592365-0801-497a-b9ba-c1b514a9355d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:50.492173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592365-0801-497a-b9ba-c1b514a9355d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:50.492348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592365-0801-497a-b9ba-c1b514a9355d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:50.517097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-97592365-0801-497a-b9ba-c1b514a9355d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:50.518315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-97592365-0801-497a-b9ba-c1b514a9355d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:50.619050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 176/363] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:50.701139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09a776-ab32-4c7d-9af6-86dc8abc0755 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:50.701475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09a776-ab32-4c7d-9af6-86dc8abc0755 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:50.701684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09a776-ab32-4c7d-9af6-86dc8abc0755 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:50.703358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 188/364] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:50.708223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:50.708509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:50.708777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:50.733104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:50.734353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:50.816004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 177/365] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:50.939299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09a776-ab32-4c7d-9af6-86dc8abc0755 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.070372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09a776-ab32-4c7d-9af6-86dc8abc0755 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.116860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.117565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.117845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.118119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.131978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:51.137412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:51.138299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:51.139073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 189/366] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:51.144653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.165850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.224720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.230164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.279216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d3b56-502a-4449-82b2-e34eb43aa186 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.297826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80160f2-c2e5-4704-a4b3-d696372e2e8a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.316013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.316767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.317043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.317325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.329292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5a72edcb-b9ea-4cf5-8c89-a05012f2d834 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:51.334519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:51.335451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:51.336294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 190/367] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:51.339368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ee0-9525-4aea-8f06-2e1500bd3b95 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.339368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ee0-9525-4aea-8f06-2e1500bd3b95 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.339368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ee0-9525-4aea-8f06-2e1500bd3b95 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.339368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 191/368] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:51.344419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b1ee0-9525-4aea-8f06-2e1500bd3b95 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.346644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.347842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.349380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.349862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.350776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:51.351282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:51.353058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7753f12-b885-4492-b46a-cf2da3f25bff None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:51.365193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 178/369] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:51.366834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e23ba-4069-440e-9695-718904bac19e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.367316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e23ba-4069-440e-9695-718904bac19e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.368389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e23ba-4069-440e-9695-718904bac19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.392579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d70e23ba-4069-440e-9695-718904bac19e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:51.394723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d70e23ba-4069-440e-9695-718904bac19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:51.458435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b1ee0-9525-4aea-8f06-2e1500bd3b95 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.463801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 179/370] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:51.464773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e23ba-4069-440e-9695-718904bac19e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.491963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.508850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.527047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.527690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.527969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.528264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.529001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:51.529304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:51.530705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:51.545890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 192/371] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:51.553509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.607229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e23ba-4069-440e-9695-718904bac19e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.671206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.671850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.672805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.673076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.691778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:51.693251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:51.719493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:51.720736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:51.721749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:51.721749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:51.721749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:51.723329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 180/372] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:51.733151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.744385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10497138-1b11-4298-a793-673f768a20d6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.796609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.797274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.797549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.797819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.809078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:51.810621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:51.826954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:51.827964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:51.828855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:51.828855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:51.828855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:51.828855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 193/373] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:51.829074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.864771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.869598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:51.877348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138228e6-0f25-4229-9617-ac67ff6e2f3f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.916525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.917196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.917559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.917829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.918474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:51.918772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:51.920686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:51.935369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 181/374] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:51.936417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:51.948387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a1549-e119-41b4-9865-334582ace700 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:51.982416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:51.983045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:51.983316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:51.983583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:51.993895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5d939d2b-f867-4d21-a7a7-32091ae8ba5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:51.999119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.000101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:52.000934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 194/375] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.001614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502980ca-1235-4ec6-bb3f-175522364312 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.001678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502980ca-1235-4ec6-bb3f-175522364312 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.002217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502980ca-1235-4ec6-bb3f-175522364312 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.003141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 195/376] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:52.003819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502980ca-1235-4ec6-bb3f-175522364312 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.040084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057ac22-ea36-413f-8a4e-fa5790426c32 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.072319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.072922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.073192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.073456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.074112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:52.074394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.075747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f26a0-a246-4411-8dc9-27eb459f7c3f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.086519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 182/377] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:52.087721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf797a-1b81-4ee8-9e2d-dcd632dc69fd None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.088004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf797a-1b81-4ee8-9e2d-dcd632dc69fd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.088328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf797a-1b81-4ee8-9e2d-dcd632dc69fd None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.089497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 183/378] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:52.090181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf797a-1b81-4ee8-9e2d-dcd632dc69fd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.095325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.103311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.106902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502980ca-1235-4ec6-bb3f-175522364312 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.139378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.139940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.140262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.140531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.148171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.149632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fcd0bdc3-1f6b-45fe-a3f0-eb210f9431a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:52.161694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.162640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:52.163571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 196/379] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.164166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f43af-601e-49c3-b3fb-3e382b800fec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.164166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f43af-601e-49c3-b3fb-3e382b800fec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.164635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f43af-601e-49c3-b3fb-3e382b800fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.182115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-825f43af-601e-49c3-b3fb-3e382b800fec None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:52.183253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-825f43af-601e-49c3-b3fb-3e382b800fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:52.191921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf797a-1b81-4ee8-9e2d-dcd632dc69fd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.225931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.226538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.226807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.227087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.227754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:52.228052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.229441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.236974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 184/380] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:52.237744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.248931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 197/381] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:52.250243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.250620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.250783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.269021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:52.270239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:52.312725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.346365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 198/382] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:52.346365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.356559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.360734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ce20-bb84-4873-8bc7-9bce0922bab1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.408357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.409718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.410227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.410633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.420680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.422837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:52.440187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.441266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 13:14:52.442406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:52.444870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 185/383] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.446103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.451214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b9f73-572b-409b-9b9e-2a618c0c5ac2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.485382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.486127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.486493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.486851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.487734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:52.488145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.491389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.528764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 199/384] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:52.529809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.567689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3562-8f82-4cd0-afed-002c7d172149 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.599846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.600373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.606605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.607327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.607665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.608276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.619220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.620698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.634291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38af3-93f1-4b7f-95e3-dc435de3dd7f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:52.634506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.635622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:52.636513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:52.636513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.636513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:52.636513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 186/385] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.636513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.672961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.673511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.673781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.674042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.674760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:52.675058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.676535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.693229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 200/386] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:52.694027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.741106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba724a-c561-4f5f-8b6b-f64c38480ff6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.774144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.774776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.775053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.775324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.785132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:52.791269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.792371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:52.793163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:52.793774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0979c1f-804a-4f17-9854-034188549ad6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.793846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 187/387] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.793846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.817766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.826252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.826886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.827156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.827422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.837295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.837968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:52.838786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:52.853370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.854286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:52.855199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:52.855199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.855199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:52.855199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 201/388] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.855199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.898408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1df9b-741a-4500-aa2a-70e365c1cfc1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.929983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.930620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.930895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.931165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.941456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.942913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:52.953792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127acb3-e293-43ea-80a4-e7107f5a759a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:52.956845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:52.957909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:52.958760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:52.958760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:52.958760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:52.958760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 188/389] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:52.958760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:52.988763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:52.989372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:52.989651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:52.989933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:52.990572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:52.990871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:52.992388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.003649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 202/390] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.003649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.042721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.053462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.059480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e82ea-db25-4df2-9a77-277da1d7d320 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.091362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.091995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.092293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.092578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.101906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750e060-a0ea-46b9-be4e-95dc7b83cefc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.102986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.104449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:53.118854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:53.119888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:53.120968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:53.120968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:53.120968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:53.120968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 189/391] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:53.125214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.135901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.136550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.136824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.137109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.137797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.138094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.139440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.147635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 203/392] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.148422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.227585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb21c5e-6964-44c3-9075-aff406fc9683 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.241996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.244831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd2d1-7133-4e32-b22a-07df6a9ab643 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.262017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.262636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.262915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.263185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.263838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.264133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.265541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.277636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 190/393] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.278390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.279938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.280627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.280916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.281186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.281842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.282151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.283506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.288905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.295702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 204/394] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.296522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.376827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96b06-4381-4ffe-855f-e108932dbac9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.399649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462a97-e1bb-468c-a85f-f9a7e379899a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.410362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.411001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.411272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.411537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.418873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.420324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:53.432556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:53.433564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:14:53.439342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:53.439987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.440085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 191/395] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:53.440182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.440182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.440182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.440399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.440485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.440485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.440485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.449860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 205/396] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.450649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.473227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.499626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.532571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ca280-d336-4e40-84f5-59c841295f5e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.549964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8df1-e6df-4c5c-948e-a7f4d2092fc0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.566818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.567440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.567715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.568230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.579431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.580926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.584443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.585188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.585466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.585735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.593988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:53.595093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:53.596163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:53.597200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:53.597200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:53.597200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:53.597406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.597485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 192/397] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:53.597485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:14:53.616619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:53.617798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 206/398] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:53.619424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.700624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.703884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a0a65-42e5-4a5b-b24f-412d2ea8fc20 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.720713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f565e4-babb-4baa-be98-ebdb520ca3a4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.724303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.739586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.740217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.740497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.740803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.741667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.744106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.755271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.755813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.756088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.756386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.757158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.757553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.758915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.768722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 193/399] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:14:53 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:14:53.770162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.791506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 207/400] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.791506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.896789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b3de1-6032-4525-bc01-82a13823c66f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.912494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388428e-8b8b-46b3-b074-d524357d22f5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:53.935988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.936826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.937200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.937593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.945035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.946760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.952233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:53.953027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:53.953376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:53.953638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:53.954349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:53.954697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:53.956061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:14:53.959559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:53.960721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:53.961758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:53.962171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 194/401] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:53.963221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:53.965600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:53.972273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 208/402] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:53.973845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.003548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.075399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b842-3882-4ad0-9437-610120c6d5cc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.081288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d17b46-d9c0-4676-ae35-1ce4ca5c5ebb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.110269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.111234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.111514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.111782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.122785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.123244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.123967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.124274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.124454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.124856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.125674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:54.126144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.127794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:54.139039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:54.140255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:54.141335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:54.141335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:54.141335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:54.141335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 209/403] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:54.144235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.150478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 195/404] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:54.151451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.197012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.255213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766f98-1e0b-4585-ba2f-018085541036 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.262558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.271016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870923b-7bae-47ba-bc6d-86e98ef3b81b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.296019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.296827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.297226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.297588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.298423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:54.298840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.300310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.309299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.310144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.310555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.310957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.320564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.321736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 210/405] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:54.322398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.322592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:54.339243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:54.341669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 196/406] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:54.347389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.442914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e91e5-c714-4500-bac9-6d54ced3dc54 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.454428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.459198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725a5a8-2baf-46bc-a234-49705f35933a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.478557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.479175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.479476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.479719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.480392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:54.480690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.482080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.493748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 211/407] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:54.494365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.500006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.500716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.500991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.501259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.519719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=edc1cd61b66547b2a388b1ead24a47c1, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.521203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.539889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 197/408] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:54 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:54.540557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.550163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.634911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769e6cd-d27f-42b7-a06d-4d50e45b3d4d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.679666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.680428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.680714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.680987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/f056f1106e9a4b5ba29fe99058e47c1c/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.682771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600334d-e901-4be8-9af4-52be9ee49b4a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.701574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=f056f1106e9a4b5ba29fe99058e47c1c, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.703350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.728770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 212/409] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:54 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/f056f1106e9a4b5ba29fe99058e47c1c/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:54.729736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.733369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.734086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.734365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.734645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.736497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:54.736857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.738476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.764539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.794910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 198/410] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:54.796922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.858581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68ac8e-fc00-430d-8ea8-03c6c738ca46 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.878770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:54.940149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.940149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.940149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.940149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.940596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405d7ea-8728-4588-8a00-ae39722a27a2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:54.945506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.947084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:14:54.960864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:54.961786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:54.963319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 213/411] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:54.973091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:54.978523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:54.980079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:54.980396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:54.980670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:54.993437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:54.994998 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:55.015732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:55.016743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:55.018497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:55.018497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:55.018497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:55.018497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 199/412] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:55.022890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:55.078679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:55.116843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429104b6-b576-4191-9c5b-ef83fdc8ffd9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.138331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341143f-f7f0-46ed-a809-61989221c9a7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.160485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:55.177183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.177803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.178082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.178357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.179074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:55.179380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.180950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.193434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.193814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 200/413] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:14:55.193987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.194322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.194594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.195297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:55.195600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.197216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.199398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:55.227523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 214/414] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:55.233540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:55.305266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0712-4165-412c-bfa4-955138bbd84e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.340317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.340958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.341236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.341509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.348673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.350128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:14:55.361853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:55.362701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:55.363979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 201/415] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:55.369003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:55.407351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:55.473674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f02945-df16-47c6-9969-03c62fe8a0a0 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.506278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f62b6-2b9b-41e9-a0aa-d358f2e028b5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.509379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.510002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.510281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.510555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.511259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:55.511563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.513037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.526525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 202/416] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:55.532424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:55.576105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.577228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.577725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.578228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/3ecc3d37e6214164a01eb96629509215/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.597216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:55.603516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=3ecc3d37e6214164a01eb96629509215, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.605373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99300ab4-eb3d-46e9-9273-b1667003cc55 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.629672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 215/417] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:55 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/3ecc3d37e6214164a01eb96629509215/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:14:55.634800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:55.641738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043350b4-8953-44d3-a083-bc93f19d80b6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:55.684078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:55.684869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:55.685117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:55.685394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/b07bb14ad0a147fda168a9165a74b7e1/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:55.704297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=b07bb14ad0a147fda168a9165a74b7e1, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:55.705760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1c164-aeb8-4609-81b7-000ea5c92bd5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:55.721893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 203/418] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:55 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/b07bb14ad0a147fda168a9165a74b7e1/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:56.240402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff980d-250d-4712-a0f8-5fed9f14da1e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:56.241450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff980d-250d-4712-a0f8-5fed9f14da1e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:56.241955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff980d-250d-4712-a0f8-5fed9f14da1e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:56.243481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 216/419] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:56.249052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:56.249334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:56.249594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:56.280107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:56.281343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:56.358146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 204/420] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:56.620864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff980d-250d-4712-a0f8-5fed9f14da1e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:56.757776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff980d-250d-4712-a0f8-5fed9f14da1e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:56.805375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:56.806021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:56.806300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:56.806570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:56.819920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:56.824870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:56.825717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:56.826964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 217/421] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:56.837364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:56.962104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:56.993939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fc106-f90b-4713-b22e-44bcf6c3de7d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.045103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:57.045724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.046003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.046296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.046985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:57.047290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:57.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb379df-738d-4715-8b75-02107b423ea5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:57.057382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 205/422] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:57.062963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:57.131415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:57.171822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cb610-61be-4b02-851c-5bf267cf984e None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.172109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cb610-61be-4b02-851c-5bf267cf984e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.172414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cb610-61be-4b02-851c-5bf267cf984e None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.174011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 206/423] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:57.179015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.179487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.179929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.207424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcac3f-7152-47eb-b931-1665fb17558e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.210018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:57.212284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:57.263906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:57.265317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.265769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.266276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.277507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:57.279236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:57.296743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:57.297719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:57.298553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:57.298553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:57.298553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:57.300403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 218/424] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:57.309331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:57.343583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 207/425] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:57.460011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d87424-6c55-493a-ae6a-24a697ab966b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.461573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:57.514726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:57.515266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.515543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.515811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.516508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:57.516804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:57.518177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89a71e-95de-41e8-86c6-0c3bb2d00088 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:57.535089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 219/426] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:57.540618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:57.608652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e54b5-0476-4d43-a13d-80269da27d6f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.608942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e54b5-0476-4d43-a13d-80269da27d6f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.609206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e54b5-0476-4d43-a13d-80269da27d6f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.610985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 220/427] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:14:57.618521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.620017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.620701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.648470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:57.656653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:57.658137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:57.701268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f37e9a-7405-4cc1-8268-a9ec46c0d343 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.737200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:57.737841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:57.738114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:57.738384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:57.745676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:57.747109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:57.757723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 221/428] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:57.764182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:57.765372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 208/429] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:14:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:57.766359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:57.767819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:57.938094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67711138-9576-47eb-94cd-faa1b41d2158 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.950563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:57.965751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d5a7-9bad-4787-b777-0d9ddd314f13 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:57.982948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:57.997257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:57.998130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.000803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.001238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.002124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:58.003120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.007859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.039054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 209/430] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:14:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:58.039936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.040761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.041195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.041653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.043086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.064935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:58.074180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.075285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:58.076341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 222/431] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:58.077979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.157467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534055a-e7a7-4780-a15c-bba180aceeb8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.184651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d294636-d034-48d6-ad09-2b8771f7c9ab None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.195490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.196597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.197124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.197621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.209531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.211489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.221980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.222638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.222911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.223176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:58.229675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.230620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:58.231528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:58.231528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.231528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:58.236509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:58.238479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 210/432] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:58.239578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:58.242951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.243888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:58.244760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:58.248489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 223/433] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:14:58.249759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.280543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.288522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.352612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3c29d-7141-489d-a640-d8ee73e15909 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.371365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8979a9-2361-432d-aed6-e7bff352db05 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.385132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.385745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.386024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.386293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.386968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:58.387276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.388677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.396461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 211/434] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:58.397576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.408967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.409580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.409852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.410123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.410773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:58.411072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.412484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da04ae9-e154-4a03-b430-79a79f83c59e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.423954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 224/435] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:58.425266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afda87-f898-408f-94db-c02e7c1dc5a5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.425549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afda87-f898-408f-94db-c02e7c1dc5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.425821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afda87-f898-408f-94db-c02e7c1dc5a5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.427041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 225/436] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:58.428147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e73e9-da59-4c2e-8e5d-2b9609ee13e0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.428424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e73e9-da59-4c2e-8e5d-2b9609ee13e0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.428686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e73e9-da59-4c2e-8e5d-2b9609ee13e0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.430011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 226/437] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:14:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:14:58.433036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e73e9-da59-4c2e-8e5d-2b9609ee13e0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.499153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.501820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.502048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08522-5786-49bc-a4ad-7cad7227635b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.533959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e73e9-da59-4c2e-8e5d-2b9609ee13e0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.536245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.536854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.537133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.537404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.538124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:58.538432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.539795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.547760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 212/438] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:58.548562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.574917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.575629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.575938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.576233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.587149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.588661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b18708ab-156c-4204-94a2-e973e105445c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:58.603320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.604288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:58.605142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:58.605142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.605142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:58.608293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 227/439] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:58.609800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c448bdd-869b-4983-ac3b-3d980623453e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.610080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c448bdd-869b-4983-ac3b-3d980623453e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.610344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c448bdd-869b-4983-ac3b-3d980623453e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.627900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9c448bdd-869b-4983-ac3b-3d980623453e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:58.629117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9c448bdd-869b-4983-ac3b-3d980623453e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:58.648255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c3f1b-bf82-4a4b-8146-763b428274ac None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.681390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.682018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.682296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.682565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.689048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.690503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.696773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 228/440] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:58.697930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.698211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.698474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:14:58.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.702711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 213/441] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:58.708651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.723788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:14:58.725171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:14:58.733733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.776424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:58.800287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 229/442] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:14:58.800287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.823781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea20bd2-a557-4bd9-abcf-0ad47777382e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.864243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.864804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.865080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.865345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.878863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.880848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:58.899040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:58.899940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:58.906150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:58.906150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:58.906150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:58.906150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 214/443] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:58.906150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:58.915678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2b60a-5d77-484a-98c1-862012fbb694 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:58.950405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:58.951042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:58.951315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:58.951612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:58.952456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:58.952764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:58.954915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:58.967723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 230/444] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:58.968560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.000315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.013060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.027039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4e66a-50a5-4433-a48b-4bee3f54bbfa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.062441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.063057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.063335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.063607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.064335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.064645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.066114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.072650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83a6cb-09b0-43d2-bc15-38d0247e0b3f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.080197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 215/445] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.081023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.105988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.106605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.106882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.107148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.107784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.108086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.109496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.121403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 231/446] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.122124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.183256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f8c8-7527-40d0-a069-155e30423c7a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.216476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea1f9b-058b-4301-a7ce-92322d502cf3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.217025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.217680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.217956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.218219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.219206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.220094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.230022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:59.234868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:59.235764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:59.236605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:59.240900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 216/447] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:59.241653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.250726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.251356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.251630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.251886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.259021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.260496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:59.272361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:59.273264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 232/448] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:59.274153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.344023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff83dc4-1073-44c0-91d5-aaf1e602aabb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.375145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047446c0-7e85-40b6-8119-e96877b7b28f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.378652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.379284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.379564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.379834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.389864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:14:59.394628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:59.395494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:59.396317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 217/449] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:59.397290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.410424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.411082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.411353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.411614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/8f89a9124ef147f9a402e7df844ed088/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.425790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=8f89a9124ef147f9a402e7df844ed088, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.427243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.440286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 233/450] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:14:59 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/8f89a9124ef147f9a402e7df844ed088/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:14:59.441379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.454438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.457746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.508378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5e90-832c-404b-81e7-b0587ed27224 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.555119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.556367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.556850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.557293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.557544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f0268-42e3-4ee2-accb-2518cb322231 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.566267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.567670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:14:59.581843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:59.582849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 13:14:59.583725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:14:59.590966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 218/451] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:59.590966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.610960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.611915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.612232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.612526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.613628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.613940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.615358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.626818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 234/452] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.626818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.695061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de10dda-8958-4699-be59-da8c3d27cbd9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.706872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.708694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.730121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.730729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.731007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.731278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.732012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.732348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.732813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca584-0107-4932-9328-70713216ea50 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.733769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.755805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 219/453] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.755805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.772687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.773414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.773686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.773948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.774721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.775124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.776520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.785363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 235/454] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.785985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.869586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8966495-4425-42f6-807e-2ac58e0b58e1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.890401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d11a-6557-40e4-8cff-0719ed573a57 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:14:59.908540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.909147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.909419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.909681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.910513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:14:59.910817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.913611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.928933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:14:59.929579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:14:59.929849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:14:59.930113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 220/455] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:14:59.930226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:14:59.930923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:14:59.941976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.946715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:14:59.949057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:14:59.961994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:14:59.964592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:14:59.965463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:14:59.966289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:14:59.966289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:14:59.966289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:14:59.970304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 236/456] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:14:59.971133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.036587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fce1b1-e1f5-4d03-a395-fb6da8648251 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.070813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.071418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.071696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.071962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.082735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceb546-c394-4320-8be0-7977bc92f842 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.117193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.117825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.118099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.118395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.129577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.131140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.140659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.142035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:00.146303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:00.147581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:00.148995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:00.148995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.148995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:00.152414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 237/457] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:00.153096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.172187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.197574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.248987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 221/458] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:15:00.254304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469c117-8686-4c60-8e50-38fbf58040af None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.254889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.294268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.294889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.295162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.295427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.358937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe518-49ec-473b-adca-187184bc1636 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.365340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.366962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.385978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.392319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.392859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.393137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.393402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.394041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:00.394344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.395694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.407110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 222/459] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:00.407701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.415324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.480814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 238/460] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:15:00.480814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.517064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f23e-a7fc-41b7-8e73-0f84d2663115 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.552010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.552648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.552926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.553191 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.554306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:00.554306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.555614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.573011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 223/461] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:00.573011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.584200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c2704-6d02-4320-9aa2-ef6b307222ac None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.622384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.622832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.623458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.623823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.624040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.634540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.635982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.637835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:00.654153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:00.655055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:00.655898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:00.655898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.655898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:00.656059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 239/462] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:00.656789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.680236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7635a7-984e-492b-9eb2-c248b9e7301b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.715679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.716756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.717033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.717458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.725953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.727501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:00.742316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:00.743208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:00.744631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 224/463] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:00.745376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.764756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c2a0-3134-4573-82c4-8ee196545387 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.798380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.799065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.799343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.799607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.810778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.812271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:00.826245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:00.827138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:00.827977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:00.827977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:00.827977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:00.827977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 240/464] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:00.829067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.851905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ddeef-aaff-40b6-82df-c9e726b6492a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.863998 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.879196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:00.889286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.889917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.890193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.890462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.898428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.899883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:00.911485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:00.912588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:15:00.914299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:00.917268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 225/465] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:00.918309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:00.936703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcfc5a-14ff-44d6-b904-8025113b382c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:00.970252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:00.970865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:00.971139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:00.971404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:00.972049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:00.972378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:00.973757 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:00.985754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 241/466] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:00.986380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.026071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b0cd-aae1-428a-bfbe-5db1b1ef8988 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.060750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.061364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.061627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.061895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.062637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:01.062940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.064405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.086013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 226/467] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:01.086365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62c414-d96d-45d3-8dd7-7551c21aec11 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.086907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.094423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.115160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.125317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.125933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.126203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.126463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.127098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:01.127479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.129300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.148679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 242/468] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:01.148679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.193553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6951b-e746-426b-a7ce-06647b4bd2f0 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.230613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.231222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.231491 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.231757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.232522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:01.232827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.234224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.251425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 227/469] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:01.252144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.257495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b14f2-da90-464a-970a-2ffd5d9c0fb7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.293478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.294118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.294390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.294658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.301773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.303207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:01.315115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:01.315958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:01.316836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 243/470] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:01.317288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.344986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.345695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.359712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32328b-eff6-4c78-bce1-75a2394c9c45 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.394017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.394633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.394911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.395179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.426887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94376a7-6374-4b72-9c00-1e584bc49543 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.466257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.466823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.467101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.467371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.470825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.472346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.474817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.476412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:01.488185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:01.489043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:01.489865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 244/471] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:01.490324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.574071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.575938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.590257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 228/472] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:15:01.591167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.592403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6370c8-93c0-46d6-b1cb-c982cfbec5a1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.626565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.627179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.627454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.627723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.689039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847077f3-3971-4a4b-9eef-af7aca7926b1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.696156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.697653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab0b74-226f-4461-8908-e4e4bcbce7cb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.723890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.724442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.724713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.724979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.725693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:01.725988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.727385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.739064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 229/473] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:01.739840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.783318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.788272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:01.798777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 245/474] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:15:01.800076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84410bfe-2e7e-4af3-a7b8-66dbc8c43100 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.800393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84410bfe-2e7e-4af3-a7b8-66dbc8c43100 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.800659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84410bfe-2e7e-4af3-a7b8-66dbc8c43100 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.802033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 246/475] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:01.802610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84410bfe-2e7e-4af3-a7b8-66dbc8c43100 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.843165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b860f-ef12-4116-bfdf-1517bbd7f1aa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.876648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.877247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.877522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.877786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.878471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:01.878774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.880190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.899674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 230/476] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:01.899674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.903910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84410bfe-2e7e-4af3-a7b8-66dbc8c43100 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:01.936596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:01.937222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:01.937490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:01.937804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:01.947499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:01.949010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:01.963501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:01.964451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:01.965331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:01.965331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:01.965331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:01.965331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 247/477] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:01.965331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:01.994217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729459-f7c6-4c49-8ddd-fb2b523a5a86 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.001306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.004854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.027386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.027984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.028284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.028551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.029280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=edc1cd61b66547b2a388b1ead24a47c1) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.029581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.030996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ac785-29a3-4efd-8b56-797f18f563a7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.036509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 231/478] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/role_assignments?user.id=edc1cd61b66547b2a388b1ead24a47c1&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.037625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.037931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.038196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.057583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:02.058793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:02.074558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2fe-a7b6-4da0-af41-c840bb1ebac9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.117218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.117875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.118153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.118418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.132546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.133984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.144219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 232/479] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:02.144219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:02.149548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:02.150468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:02.151445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:02.151445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:02.151445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:02.151445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 248/480] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:02.151799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.272634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ce71e-c885-4c12-92f3-8740d63cc140 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.274213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.275088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.288409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850c2d-f7f3-4f2a-87c2-25dc38093b99 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.306772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.307526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.307902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.308298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.309128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=f056f1106e9a4b5ba29fe99058e47c1c) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.309532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.311059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.317581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 233/481] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/role_assignments?user.id=f056f1106e9a4b5ba29fe99058e47c1c&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.318558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.328564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.329208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.329481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.329746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.330466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.330764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.333207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.348689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 249/482] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.354866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.476419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e995b3-5176-495d-ae51-60d516cbb9e6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.514460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb79ac-d195-4472-9a60-b662ed109d9c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.521244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.522363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.522693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.522907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.523556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.523916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.525432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.541854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 234/483] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.544943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.563044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.563692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.563966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.564287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.572707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.574182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.578105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:02.594455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:02.595394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:02.596252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:02.597608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 250/484] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:02.597608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.613086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.680350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbb202-77a2-4be0-9862-5b7c3649ccbe None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.715714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.716434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.716716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.716984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.728856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:02.732772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0408-120a-4542-bc0a-1e267275c8f6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:02.735797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:02.736692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:02.737940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 235/485] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:02.738589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.783808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.784402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.784680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.784948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.795191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.796758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:02.808729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:02.809645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:02.810458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:02.814780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 251/486] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:02.815501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.843309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.870466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c65d9-aba7-4137-b13b-c3bfd5f9916b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.886264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:02.919641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333b370-625d-4330-a6c5-a6194b7a27c1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:02.921751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.923467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.923948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.924405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.925269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.925748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.927835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.945054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 236/487] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.946162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:02.960887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:02.961670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:02.962011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:02.962347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:02.963130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:02.963498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:02.965025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:02.973704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 252/488] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:02.974653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.071130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28ba86-f01e-464b-bdc9-8790c6d4fa08 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.088021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07b6d-5fae-4d54-9c17-74d1b7890ee5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.108782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.109689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.110151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.110586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.111952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:03.112454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.114526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.121392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:03.127451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.128227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.128610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.128950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.129929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=b07bb14ad0a147fda168a9165a74b7e1) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:03.130349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.131908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.133436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 237/489] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:03.134437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.138068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:03.144392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 253/490] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:02 2022] GET /identity/v3/role_assignments?user.id=b07bb14ad0a147fda168a9165a74b7e1&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:03.145849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.273032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98ab0-0eb8-4720-bd49-c440cdd6ac96 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.315724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.316767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.317268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.317948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.319120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=3ecc3d37e6214164a01eb96629509215) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:03.319648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.322159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.330234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 254/491] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:03 2022] GET /identity/v3/role_assignments?user.id=3ecc3d37e6214164a01eb96629509215&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:03.347513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0543-4b00-4b9a-b4d4-21dea83d9b99 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.393782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.394431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.394708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.394975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.407816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.409515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:03.425826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:03.426781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:03.428166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:03.428166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:03.428166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:03.428166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 238/492] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:03.433172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.503044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:03.552042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb5978-f7fb-4f2e-80b0-63834c8f43ae None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.587644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.588313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.588655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.589137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.589734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:03.590097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.591603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.604735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 255/493] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:03.610040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.680579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:03.727889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9231-c30d-4ca1-b3f1-9b9c1fad29fc None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.771163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:03.771804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:03.772451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:03.772718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:03.789288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:03.790733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:03.806710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:03.807656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:03.808605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 239/494] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:03.813695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:03.969778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e3c73-9c2c-4731-9aa9-c245f7c3ebd2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:03.998982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:04.020780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:04.021401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.021673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.022367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.023112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:04.023412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:04.024864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:04.052756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 256/495] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:15:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:04.058443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:04.215973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2b3e7-89e9-4c23-9255-1c688264f7e8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:04.237873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:04.257937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:04.258555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.258830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.259103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.349757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:04.351317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97c76-c9d1-4ac0-9446-f8b0883dfc0f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:04.501478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 240/496] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 13:15:04.507250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:04.600213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596673-687f-4587-8800-00bd3073a5c8 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.600573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596673-687f-4587-8800-00bd3073a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.600844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596673-687f-4587-8800-00bd3073a5c8 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.602417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 241/497] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:04.607343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23750d-a0d8-4bc5-8868-b113ab4a3c9d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.607626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23750d-a0d8-4bc5-8868-b113ab4a3c9d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.607906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23750d-a0d8-4bc5-8868-b113ab4a3c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.626567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-4f23750d-a0d8-4bc5-8868-b113ab4a3c9d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:04.627819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4f23750d-a0d8-4bc5-8868-b113ab4a3c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:04.641786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:04.687139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ef31-c3fc-4a4c-84f4-b9a275d37a4f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:04.711656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 242/498] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:15:04.749249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:04.749947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.750230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.750499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.761027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:04.762413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:04.777515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:04.778466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:04.779340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:04.779340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:04.779340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:04.779340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 257/499] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:04.780305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562623-bdaa-45cf-8767-5bccb5cce7fc None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.780787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562623-bdaa-45cf-8767-5bccb5cce7fc None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.781243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562623-bdaa-45cf-8767-5bccb5cce7fc None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.782966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 243/500] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:04.786122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:04.788690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:04.789205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:04.789664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:04.807641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:04.809197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:04.828576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:04.938139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 244/501] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 13:15:04.954286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a9c2c-197f-4b0b-8d0f-118b40b34ead None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.002638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.003259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.003537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.003808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.004489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:05.004792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.006183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.025437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 258/502] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:05.028708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.040589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.164493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.200232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd622-3afb-4113-b7ac-3a4666a36108 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.220331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.227781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fedf1-9cac-4286-a0b1-aba449cf6325 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.234385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.235025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.235297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.235564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.245720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:05.250517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:05.251874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:05.252759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 245/503] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:05.253443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.267926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.268945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.269519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.269977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.277513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.279289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:05.291292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:05.292241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:05.293207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:05.295461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 259/504] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:05.296534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.353358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42702e03-e0f8-4460-990b-6739431a245c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.387030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.387775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.388146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.388513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.391444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.400653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.402225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.404395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4500d-d913-4cf5-a202-35ccbdaacfef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:05.416799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:05.417755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:05.418616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:05.418616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:05.418616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:05.418616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 246/505] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:05.424269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.452773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.453614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.453943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.454214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.454936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:05.455230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.457458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.470119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 260/506] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:05.471512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.491093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.543473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f981f-6911-48a9-8436-23dfef386ccc None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.587336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.588488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.588934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.589439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.590395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:05.590839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.592680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.600157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f616-7f16-4eb1-bc17-6c3f40d117d6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.608740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 247/507] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:05.610235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.643864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.644978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.645354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.646173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.647579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:05.648036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.650441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.669561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 261/508] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:05.670814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.672993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.729247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead3de-d084-4109-a35a-f562b1ab94c7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.742187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.765871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.766513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.766791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.767062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.775936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc473fa-a8d6-4f84-a6a9-d57443734942 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.778160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.779673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:05.794989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:05.795992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:05.796919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:05.796919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:05.796919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:05.796919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 248/509] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:05.797150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.815426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.816528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.817034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.817680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.818637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=8f89a9124ef147f9a402e7df844ed088) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:05.819234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.821124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.828675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 262/510] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/role_assignments?user.id=8f89a9124ef147f9a402e7df844ed088&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:05.829917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:05.907107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:05.915489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cd30-e8c3-4d65-8079-a880bc17454c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:05.985078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:05.985703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:05.985980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:05.986252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:05.986909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:05.988501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ae891-6a78-47ab-9019-e8f36c458afc None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:05.995894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee14f6-7f46-4be1-9ddc-36827f1a37f0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.014360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 249/511] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:15:05 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:06.016176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d942-9662-49bd-b3cc-4e262bb0e812 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.016176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d942-9662-49bd-b3cc-4e262bb0e812 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.016765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d942-9662-49bd-b3cc-4e262bb0e812 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.018331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 250/512] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:06.025180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655632-875f-4561-89e1-b2ccd73cf13c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.025736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655632-875f-4561-89e1-b2ccd73cf13c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.026150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655632-875f-4561-89e1-b2ccd73cf13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.048573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fd655632-875f-4561-89e1-b2ccd73cf13c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:06.050213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fd655632-875f-4561-89e1-b2ccd73cf13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:06.064845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.065489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.065918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.066285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.067076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:06.067498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.069195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2facc-b43f-4739-8721-6a06d4049833 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.075144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.090155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 263/513] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:06.090488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505d17-37d3-4325-aaff-528d271fced8 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.090769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505d17-37d3-4325-aaff-528d271fced8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.091093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505d17-37d3-4325-aaff-528d271fced8 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.093164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 264/514] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:06.095454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96505d17-37d3-4325-aaff-528d271fced8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.155610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 251/515] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:06.158228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.158510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.158774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.177786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:06.179064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:06.226035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96505d17-37d3-4325-aaff-528d271fced8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.256979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 252/516] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:06.269920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.270575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.270908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.271178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.285016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.286488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:06.305735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:06.306679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.308218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 265/517] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:06.312522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.343988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.440467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21847eb-e60f-4cf7-8a3a-1ff799b6b017 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.465839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.474548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.475027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.475371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.475647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.476469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:06.476781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.478220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.500240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 253/518] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:06.501743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.597307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.644507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.653105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a94ef-05eb-4978-b277-330cd5edf61d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.654259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0d6f4-78f2-4077-9101-23adbfef4579 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.687655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.688469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.688840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.689253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.689945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.691625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.692488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.693199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.693645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.694026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:06.703019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:06.704292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:06.705310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.706500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.708003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.710508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 266/519] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:06.711484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:06.727002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:06.728202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:06.729325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:06.729325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:06.729325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:06.735953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 254/520] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:06.736898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.839534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af1d1-f2d4-4591-861a-759772a27d5e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.851427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4b99e-0983-4bd6-8985-4695f7ae268c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:06.861063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.877201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.877608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.877886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.878151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.878738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.880137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.884633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:06.890405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:06.891409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:06.892283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:06.896528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 267/521] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:06.897723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:06.897862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:06.898683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:06.899171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:06.899638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:06.900514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:06.901340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:06.903937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:06.910544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 255/522] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:06.911579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.025490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441558-d572-4aab-aee0-df6131a38cc3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.033313 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69ae4c-2e5b-4a95-b23f-c715ca48ab2b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.071563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.073123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.073406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.073674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.074343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:07.074642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.076080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.080772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.081398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.081674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.081948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.082640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:07.082941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.084373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.092331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 256/523] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:07.093403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.100257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 268/524] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:07.100257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.150216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:07.163913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:07.213999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17f8e-3ad5-49e4-be58-2a3d701f3b51 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.214898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a21d6-8409-4ecd-832d-ac7bc038c8a7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.253649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.254440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.254687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.255100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.255316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.255647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.255754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.256105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.256718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:07.256979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:07.257314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.257431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.258902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.259357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.269662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 269/525] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:07.269662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.269934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 257/526] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:07.402351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86e1b-bda1-4914-bad2-c81e99836c73 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.413348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:07.441844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.442564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.442836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.443098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.451410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.453041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:07.472375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:07.473403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:07.474307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 270/527] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:07.479820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.609526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c93ec-9583-4f4c-9b59-156bff706ee4 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.628379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:07.646463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.647427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.647706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.647978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.648716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:07.649023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.650517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ab170-ed9d-4dac-8e6d-ed97bb032f8f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.663329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 258/528] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:07.669070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:07.788499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5440e-c653-4478-8b10-07d882597a94 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:07.834001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:07.834756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:07.835091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:07.835369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:07.837305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=edc1cd61b66547b2a388b1ead24a47c1, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:07.838749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebeb1-573a-44d9-99d9-eae94627fb41 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:07.863973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 271/529] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:07 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:15:08.282869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f176d3-c025-4473-8d34-3b0ba7a74d97 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:08.283157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f176d3-c025-4473-8d34-3b0ba7a74d97 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:08.283434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f176d3-c025-4473-8d34-3b0ba7a74d97 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:08.285076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 259/530] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:08.294099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5741-fb85-46e8-b7b6-5a147605b17f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:08.294656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5741-fb85-46e8-b7b6-5a147605b17f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:08.295127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5741-fb85-46e8-b7b6-5a147605b17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:08.325368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a10b5741-fb85-46e8-b7b6-5a147605b17f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:08.326963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a10b5741-fb85-46e8-b7b6-5a147605b17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:08.358573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:08.418386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 272/531] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:08.468982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628595e-4a55-4705-b499-36c9f8b22f48 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:08.469297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628595e-4a55-4705-b499-36c9f8b22f48 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:08.469531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628595e-4a55-4705-b499-36c9f8b22f48 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:08.471156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 260/532] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:08.476393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:08.476685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:08.476960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:08.504224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:08.505435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:08.592837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 273/533] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:08.795088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628595e-4a55-4705-b499-36c9f8b22f48 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:08.904492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:08.919882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628595e-4a55-4705-b499-36c9f8b22f48 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:08.972127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:08.972783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:08.973058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:08.973327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:08.995644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:09.000503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:09.001413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:09.002258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:09.007380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 261/534] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:09.013002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.037746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195abf2c-0124-4e9e-964d-4013be780652 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.046761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.056051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.100480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.101133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.101406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.101672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.113548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:09.120079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45405b-d41f-4980-9ba6-2a7b9d192c84 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:09.128718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:09.129686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:09.130505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 274/535] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:09.137454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.163231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.163762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.164083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.164413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.165090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:09.165394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.166873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.184731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 262/536] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:09.190328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.292607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3f1e7-1377-480d-b332-7e236d7dfc2c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.345593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.349138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.349431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.349703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.350412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:09.350717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.352111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.369651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 275/537] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:09.369955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.378510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4a5ca-3645-49ef-be1b-dc9eac8806cf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.392175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.406247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.441506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.442156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.442434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.442705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.465481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.467067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:09.490849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:09.491912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:09.492839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:09.492839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:09.492839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:09.496489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 263/538] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:09.501183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.506882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da3ab-0ee3-4e47-9d9c-ee3ffd7c0677 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.549549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.550203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.550480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.550997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.563909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.566143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:09.582745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:09.583726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:09.584690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:09.584690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:09.584690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:09.584690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 276/539] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:09.590994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.704573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.719019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6fa2-53a1-4cc7-ad37-bd368559cf3c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.755537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.756917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.757239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.757513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.758220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:09.758526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.759903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.774402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 264/540] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:09.782503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.787106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4105543-01a4-454b-888d-2dbbe650d7ca None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:09.808669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:09.851665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:09.852324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:09.852603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:09.852872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:09.853534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:09.853836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:09.855232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:09.876551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 277/541] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:09.883375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:09.967086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46507cfc-e1ea-45b8-a1a8-2cf58be78ba7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.020896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.021535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.021811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.022462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.029820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.031262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.036317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca97606-af60-467b-afd2-d1ca0ae09c0e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:10.043165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:10.045274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:10.047450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 265/542] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:10.054367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.072680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.092985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.093642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.093980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.094255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.104332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.106078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-37c792e8-1852-4763-add0-2c3722f8c13c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:10.121529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:10.123703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 278/543] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:15:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4097df-9514-4ebb-83c1-eab5905aeedc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.125904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4097df-9514-4ebb-83c1-eab5905aeedc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.127533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4097df-9514-4ebb-83c1-eab5905aeedc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.127533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 279/544] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:10.136171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4097df-9514-4ebb-83c1-eab5905aeedc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.150745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.182138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bbf4-2b66-4bd3-98ab-bad343d7e884 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.221361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.222120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.222392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.222667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.223426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:10.223731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.225252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33c86b-0e03-46e9-a294-31461ead71a9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.248764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 266/545] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:10.250439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775beba0-2fc8-4f6d-82a2-0ffe9be56b2c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.250906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775beba0-2fc8-4f6d-82a2-0ffe9be56b2c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.251356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775beba0-2fc8-4f6d-82a2-0ffe9be56b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.266633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4097df-9514-4ebb-83c1-eab5905aeedc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.271563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-775beba0-2fc8-4f6d-82a2-0ffe9be56b2c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:10.273338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-775beba0-2fc8-4f6d-82a2-0ffe9be56b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:10.310666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.311384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.311730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.312083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.312961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:10.313341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.314855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.335732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 280/546] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:10.336858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.353698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 267/547] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:10.355109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a12f7-6608-4e21-bdb9-6ed8ecc2b180 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.355458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a12f7-6608-4e21-bdb9-6ed8ecc2b180 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.355806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a12f7-6608-4e21-bdb9-6ed8ecc2b180 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.357252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 268/548] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:10.358095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a12f7-6608-4e21-bdb9-6ed8ecc2b180 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.399205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.427245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.473778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a12f7-6608-4e21-bdb9-6ed8ecc2b180 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.476007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3c987-f59c-4f88-bf43-be6f7ddd698b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.510138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.510774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.511045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.511314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.512932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.513572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.513847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.514117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.521687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.523473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.524776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.526211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-87db959b-5102-4277-93c5-e114b3691248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:10.538332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:10.539425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:10.540582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:10.540582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:10.540582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e207b102-6bcc-482e-8762-970085b3c601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:10.543295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:10.544354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 281/549] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.545226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.545567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 269/550] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:10.545636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.545710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab1e97-8722-43f6-b46c-e0fcecbb2ce1 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.545710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab1e97-8722-43f6-b46c-e0fcecbb2ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.545830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab1e97-8722-43f6-b46c-e0fcecbb2ce1 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.548161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 270/551] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:10.548161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab1e97-8722-43f6-b46c-e0fcecbb2ce1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.567050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:10.568293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:10.635768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 282/552] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:10.641042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.665760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.673949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab1e97-8722-43f6-b46c-e0fcecbb2ce1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.687797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.712185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.712856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.713131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.713397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.714140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:10.714443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.715912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ff85f-b109-4885-a6b7-f5d4fe35fca9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.732903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 271/553] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:10.732903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.732903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.732903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.748275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:10.748658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8550-5811-42db-bf57-fc216d3940d5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.749451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:10.785242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.785855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.786130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.786392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.787013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:10.787314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.788739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.804236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 283/554] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:10.804236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.826201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 272/555] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 13:15:10.827582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:10.892428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.912536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeed51f-997e-41a0-91df-cdd67662f403 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.944030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74686-5038-4c79-8b05-f01fd3708b44 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:10.945539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:10.945795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:10.946420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:10.946693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:10.946964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:10.947612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:10.949201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:10.966906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 284/556] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:10 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:10.967732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.011808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.012847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.013350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.013856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.027840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.029766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:11.047755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:11.048739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:11.049646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:11.056455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 273/557] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:11.056455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.098763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fe65b-12e3-4572-bbc1-b618f5a722b6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.144606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.145232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.145509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.145777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.154027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.155475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:11.167487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:11.168441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:11.169428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 285/558] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:11.171296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.173070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c2911-0e31-41e0-9410-fa916995754e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.175107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.211783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.212429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.212712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.212983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.213642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.214996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.232974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 274/559] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:11 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:11.233885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.260354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.290538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456b4c-3bc4-4ce6-9b15-5b44d97565b2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.334337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.335351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.335809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.336297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.337196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:11.337694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.339383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.362415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 286/560] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:11.363837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.367743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab4aa3-9934-4948-8d23-eef9c64c1b3b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.414770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.415523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.415806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.416080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.428104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:11.433338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:11.434276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:11.435126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:11.439438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 275/561] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:11.440526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.450620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.501266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f952-d9a4-4c1b-af60-0d9f0f1200a8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.536610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.547576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.548804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.549280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.549865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.551350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:11.551872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.554469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.558710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04a63-e733-455c-b98c-7c0276e363e1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.571943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 287/562] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:11.573223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.598761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.599435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.599715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.600388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/70eff2211f2741b1b3fc3f9d0c660d60/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.616485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=70eff2211f2741b1b3fc3f9d0c660d60, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.619090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.635941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 276/563] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:11 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/70eff2211f2741b1b3fc3f9d0c660d60/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:15:11.636833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.716258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae37-8a82-4529-810e-dbd4a46c466b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.750068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d98cb-aa90-47c4-8881-0279ba812798 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:11.751473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.780883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:11.781381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.782905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.783448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.783956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.793280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:11.794098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.795618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.802627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:11.803580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:11.803859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:11.804148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/8f89a9124ef147f9a402e7df844ed088/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:11.808648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=8f89a9124ef147f9a402e7df844ed088, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:11.812106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 288/564] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:11.812469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:11.818775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:11.852703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 277/565] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:11 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/8f89a9124ef147f9a402e7df844ed088/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:15:11.981431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00491f0d-f079-4c5f-b06d-bf4458c59496 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:12.021824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:12.023125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:12.023548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:12.023961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:12.040533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:12.042313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:12.062276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:12.063236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:12.064163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:12.064163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:12.064163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:12.064163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 289/566] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:12.069250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:12.184014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0e161-3f72-4cc2-a261-e06f8d371699 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:12.241988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:12.242604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:12.242884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:12.243156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:12.243816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:12.244144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:12.245702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:12.263211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 278/567] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:12.269669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:12.312766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:12.394858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe83b-1359-4f92-9295-2256d922a470 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:12.432848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:12.433486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:12.433757 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:12.434019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:12.446542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:12.447986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:12.463501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:12.464516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:12.465578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:12.466125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 290/568] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:15:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:12.470610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:12.531781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:12.594241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef40690-8c8f-471b-b870-91a707a7d506 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:12.639657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:12.640668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:12.641176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:12.641655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:12.642602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:12.643110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:12.644774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:12.667287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 279/569] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:15:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:12.672975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:12.798563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:12.809822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c8c89-3c6f-40c0-b4c8-4ae219ed5d79 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:12.860213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:12.861215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:12.861724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:12.862193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:12.947537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:12.949466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:13.059488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:13.069999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 291/570] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:13.075198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:13.178986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5eb1-7e89-4085-a9a7-23ef74203567 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:13.186972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:13.214558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:13.215534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.216040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.216538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.227151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:13.228888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:13.244007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:13.245056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:13.245948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:13.245948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:13.245948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:13.247985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 280/571] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:13.253197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:13.406118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ac096-1ce7-4efe-8bcc-dda4f18ebcd1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:13.445703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:13.446250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.446530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.446796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.447462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:13.447763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:13.449241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892b623-0d47-45fe-abfd-1fa4de5c89dc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:13.462404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 292/572] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:13.467503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:13.551430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eaf22-fb32-4f2d-a178-845e00d9559f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.551762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eaf22-fb32-4f2d-a178-845e00d9559f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.551976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eaf22-fb32-4f2d-a178-845e00d9559f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.553705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 293/573] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:13.558651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a95c3-f667-433f-b12d-45daf94d066c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.558777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a95c3-f667-433f-b12d-45daf94d066c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.559018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a95c3-f667-433f-b12d-45daf94d066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.608416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-847a95c3-f667-433f-b12d-45daf94d066c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:13.610399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-847a95c3-f667-433f-b12d-45daf94d066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:13.612598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac177b1-d8f2-426b-b86d-0d781e66b6c5 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:13.669420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:13.671200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.672446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.673193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.686444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:13.688192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:13.703980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:13.704916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:13.705812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:13.709509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:13.714396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 281/574] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:13.722909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:13.742823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 294/575] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:13.763718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:13.909538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f140f4-3e9f-4a84-996f-13f8de987547 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:13.955725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:13.956296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.956582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.956860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.957569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:13.957879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:13.959329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:13.972947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a65f-fe76-47b0-b0c7-9efcae7ca8b2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.973537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a65f-fe76-47b0-b0c7-9efcae7ca8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.973999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98a65f-fe76-47b0-b0c7-9efcae7ca8b2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:13.976026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 295/576] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:13.980262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 282/577] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:13.984768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:13.993764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:13.994253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:13.994671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.029287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:14.030756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:14.068515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:14.088716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286589-6582-496f-901b-b78179d3cc2b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:14.131410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:14.132397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.132824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.133258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.134152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=edc1cd61b66547b2a388b1ead24a47c1) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:14.136073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:14.137697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:14.154513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 283/578] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:13 2022] GET /identity/v3/role_assignments?user.id=edc1cd61b66547b2a388b1ead24a47c1&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:14.170038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 296/579] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:14.191924 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.311261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cc850-683d-4e78-a01f-5126b3a03c85 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:14.358006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:14.358925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.359351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.359767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.360541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:14.362101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:14.374096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:14.375381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:14.376482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.379558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 284/580] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:14.385668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.458398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.470773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:14.546852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4366-d14a-4ab2-b51c-333cc1c177d1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:14.562677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:14.590039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:14.590662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.590939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.591207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.591848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:14.592183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:14.593981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d470ba-99fd-418d-b29a-2ca4f325a7ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:14.597748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08631b-8e4b-4d06-a1e4-9d41fc09c342 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:14.598331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 297/581] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:14.599385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3431e-0ec5-4c0c-ae71-0adf68230731 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.599667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3431e-0ec5-4c0c-ae71-0adf68230731 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.599940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3431e-0ec5-4c0c-ae71-0adf68230731 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.601629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 298/582] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:14.602667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae16c-69aa-499a-9771-a749c1646fcd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.602946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae16c-69aa-499a-9771-a749c1646fcd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.603218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae16c-69aa-499a-9771-a749c1646fcd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.608383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 299/583] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:14.608383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3006f-aac8-4497-8e02-ff513fc2d6cf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.608383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3006f-aac8-4497-8e02-ff513fc2d6cf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.608383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3006f-aac8-4497-8e02-ff513fc2d6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.625024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f3b3006f-aac8-4497-8e02-ff513fc2d6cf None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:14.626203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f3b3006f-aac8-4497-8e02-ff513fc2d6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:14.640452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:14.641716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.642196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.642472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] PATH_INFO: `/v3/services/network` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.643060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:14.644455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:14.653601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:14.654650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:14.655554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 285/584] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:14.656408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.705566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 300/585] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:14.706641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.706929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.707203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.725244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:14.726427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:14.746775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:14.789781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634bead-4418-48cc-8e2b-0bcc6ce2b76d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:14.796506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 301/586] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:14.797415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.833561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:14.836556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:14.837544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:14.838074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:14.839068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:14.839961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:14.840490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:14.842466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:14.849784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 286/587] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:14.860242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:14.946710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0ed70-0cee-4b6e-b07d-7fbee867b9f1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.011592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.012283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.012566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.012838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.013476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] RBAC: Adding query filter params (name=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.013776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.016050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.023934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 302/588] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.032161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.044963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4165ae-cb3e-4bdf-b247-c4f751bb5e6a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.087385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.087924 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.088243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.088521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.089241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] RBAC: Adding query filter params (interface=public, service_id=f01e0b22ed534f909eac082123a7b5e6, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.089773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.090927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.092821 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.104650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 287/589] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:14 2022] GET /identity/v3/endpoints?service_id=f01e0b22ed534f909eac082123a7b5e6&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.104650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.154413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.157951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d3f-075b-42b2-9a23-7e2e9c147553 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.192005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.192642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.192919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.193187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.193870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.194177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.195554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.200387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 303/590] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.201223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.221655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adf4d9-ec49-4edc-a5dd-58b2b4642471 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.258649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.259484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.259922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.260184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.273112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:15.278319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:15.279278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:15.280145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 288/591] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:15.280854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.311204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059ccf-a5d3-4db5-ac67-68a64c98414b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.342466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.345750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.346413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.346682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.346949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.357078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:15.362102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:15.363230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:15:15.364302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:15.368210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 304/592] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:15.368927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.388383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073e846-f2a4-4485-870e-43c66c28ce6e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.394329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.426452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.427337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.427619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.427897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.428595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.430468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.435746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 289/593] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.436537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.482723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c062b-8ee6-49dd-af24-ac1b21dcaaec None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.525924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.526807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.527224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.527647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.528544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] RBAC: Adding query filter params (interface=public, service_id=a1a246d2ba3b494ca27bdcf1acea07a3, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.528982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.530600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.544066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 305/594] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/endpoints?service_id=a1a246d2ba3b494ca27bdcf1acea07a3&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.545330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.576330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9fcaf-5eb7-40fe-90dc-4782572ee193 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.606141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.622918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.623837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.624152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.624432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.625123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.625598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.627661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.639154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 290/595] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.640066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.659995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.674366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd526e-9670-4d1a-942b-2d04c8276181 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.716383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.717505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.718039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.718610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.719478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:15.719981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.721802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.731101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 306/596] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.732254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.778011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b06eb2-405a-47d7-b992-3dfc40c418ff None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.822016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.823213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.824485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.824791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.825639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.827677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.836197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 291/597] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:15.837291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:15.850579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6574b0a-44e7-4e4b-8e34-12ae55a27047 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:15.924991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:15.926059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:15.926599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:15.927066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:15.945419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:15.948454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:15.950358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:15.982758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:15.983770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:15.984843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:15.984843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:15.984843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:15.984843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 307/598] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:15.993294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.000621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.010335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f213-6fb0-4482-90fc-857bbda8a461 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.049907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.050589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.050862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.051130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.063820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.065376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:16.081502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:16.082544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:16.083468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:16.083468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:16.083468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:16.083468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 292/599] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:16.090184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.162554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b705e3-6f90-488d-b900-036f2a6d8c62 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.201936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.202941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.203450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.203786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.204537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:16.204902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.206354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.207272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.211981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a66e9-8700-4726-9818-43acd00f86d1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.220832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 308/600] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:16.226064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.264002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.265294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.265773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.266253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.267442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:16.269044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.270707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.287218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 293/601] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:16.292454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.306460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.370064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2532f-d4f4-43b5-96c7-2a0189e1dcc2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.454253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695bde4-9eb9-481b-80a2-74c36fc857fa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.459117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.460066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.460646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.461112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.492971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.495838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.500447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.501266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.501722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.502150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.517975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.519635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:16.523713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:16.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 13:15:16.525687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:16.531215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 309/602] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:16.536087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:16.553695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:16.554925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:16.556104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:16.566818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 294/603] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:16.568052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.613590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.724316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec1b49-b8a1-4c19-9820-6c40652c9ede None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.760295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.760912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.761193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.761874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.762638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:16.764146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.764707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61fc20-d368-4573-967c-f6aaa3ab06e8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.783957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 295/604] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:16.790119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111d043-d598-457b-97f6-f35e01490b0f None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.790413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111d043-d598-457b-97f6-f35e01490b0f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.790691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111d043-d598-457b-97f6-f35e01490b0f None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.792209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 296/605] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:16.792623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d0cba-2f5b-4e72-a3be-6d189429dca6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.793050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111d043-d598-457b-97f6-f35e01490b0f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:16.836954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.852786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:16.853580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.853958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.854323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.855166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:16.855567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:16.857131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdfe85-3efc-41d8-b1d3-14196c20a7b6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:16.907367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:16.923477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 310/606] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:16.923477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:16.923477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:16.923477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:16.948056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111d043-d598-457b-97f6-f35e01490b0f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:16.963256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:16.965548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:17.018101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.020660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.022668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.023563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.096327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 311/607] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:17.097484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.125669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.127260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.204069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46420aaf-985b-4063-a1c7-99426890bb73 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.215784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.243063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.244111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.244585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.245125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.262109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.305991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 297/608] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:17.311090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.322708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.324427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.429902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 312/609] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:17.435690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.457014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.475465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aed496-5ea5-4f56-97c0-0888d5f44dbb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.523086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.524181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.524662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.525178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.539790 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.545832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:17.565687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:17.566669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:17.567567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:17.567567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:17.567567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:17.569441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 298/610] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:17.570272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.602883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b882f1-1d6d-4264-82db-1bdb8393172e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.605109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.643281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.644345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.644928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.645200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.656330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.657883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:17.673656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:17.674637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:17.675525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:17.675525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:17.675525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:17.679620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 313/611] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:17.680648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.684230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ed06e-ac18-42ce-b0cc-487d41f94947 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.719475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.720131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.720467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.720749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.721338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.722709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:17.731608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:17.732572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:17.733456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 299/612] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:17.734239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.751002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.800142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7250f-b099-47af-9e47-f8ec675b05dc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.837506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:17.838986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.839887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.840396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.840872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.841677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:17.842194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.843728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.844423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa18a6-c774-4543-ac8d-2f805e2e31c6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.858431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 314/613] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:17.859617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.884695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:17.885324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:17.885605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:17.885879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:17.886538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:17.886842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:17.888250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:17.905018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 300/614] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:17.905018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:17.966060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01e21-7ad4-4971-aaf0-bbe5b246a563 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:17.991230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.004508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.005133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.005414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.005687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.006435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.006740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.008322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0a003-405a-4c6e-8148-59f080f38c73 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.010818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a202f-10b9-48ec-9009-7a2f7150bca8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.017081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 315/615] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.018484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d640650-ca99-4876-b1c1-fe85644e7bd7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.018769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d640650-ca99-4876-b1c1-fe85644e7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.019041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d640650-ca99-4876-b1c1-fe85644e7bd7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.024155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 316/616] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:18.024155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d640650-ca99-4876-b1c1-fe85644e7bd7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.056770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.058177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.058466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.058739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.066741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.068383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:18.080596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:18.081781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:18.083047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:18.086495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 301/617] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:18.087341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.104357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.128779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d640650-ca99-4876-b1c1-fe85644e7bd7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.169014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.169996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.170254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.170549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.179203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.181093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.192557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a089f5-c119-4747-94dc-988ba18aeef9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-95aad2aa-93a0-4419-a73c-6bb152eb503a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:18.193069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:18.194036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:18.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:18.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 317/618] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:18.200188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.200472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.200746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.218461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:18.219624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:18.230707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.231549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.231981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.232435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.233375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.233830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.235999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.240982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 302/619] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.241593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.290172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.320744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 318/620] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:18.321616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.345652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.376156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1122cd-8356-4471-b01f-4477b0fa83f1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.419859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.420524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.420806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.421079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.421793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.422102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.424056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3804dc-9d53-424f-a165-15bfced063eb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.438568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 303/621] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.439110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734d5f-94b7-4e7a-a764-6f681b177479 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.440604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470617-b1c5-4cb4-92e2-42abb3425266 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.441081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470617-b1c5-4cb4-92e2-42abb3425266 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.441554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470617-b1c5-4cb4-92e2-42abb3425266 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.443482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 304/622] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:18.448512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91470617-b1c5-4cb4-92e2-42abb3425266 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.482088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.482088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.482088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.482088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.483059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.483059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.484494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6aeafb-33dd-4db3-b632-939de1930b94 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.497165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 319/623] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.498662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.498816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.499039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.517642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:18.518838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:18.549288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.564859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91470617-b1c5-4cb4-92e2-42abb3425266 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.591292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 320/624] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:18.592183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.624481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.625157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.625436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.625709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.626512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=70eff2211f2741b1b3fc3f9d0c660d60) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.627056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.629514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.639438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 305/625] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/role_assignments?user.id=70eff2211f2741b1b3fc3f9d0c660d60&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.653906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.663507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.802370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae8e7-be64-4965-86ea-55b359196499 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.822122 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb37de-8ae3-44cc-9c17-0604b5f4fac7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:18.850561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.851556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.852022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.852670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.853692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=8f89a9124ef147f9a402e7df844ed088) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:18.854207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.855924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.868147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:18.868147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:18.868147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:18.868147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:18.868586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:18.868586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:18.869074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 321/626] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/role_assignments?user.id=8f89a9124ef147f9a402e7df844ed088&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:18.877634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:18.878690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:18.879944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 306/627] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:18.885314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.914028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:18.924903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:18.997557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:19.016681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b26a57-fda0-4e9e-a0a6-380727b60a1e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.044973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e5e01-c733-4a02-916e-834dfb3168e5 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.055836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.056505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.056885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.057160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.057799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:19.058102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.059590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.067422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 322/628] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:19.072246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:19.087085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.087741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.088028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.088331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] PATH_INFO: `/v3/services/network` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.088926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.090419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:19.100150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:19.101242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:19.102095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 307/629] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:18 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:19.107576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:19.181192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:19.189306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25e29-32de-412f-8f5f-3fa536dfa24b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.219779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff452751-bd45-4657-a7ae-ecb94ed49054 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.225108 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.225948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.226394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.226899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.227696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:19.228253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.229808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.236342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 323/630] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 11 13:15:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:19.242330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:19.267829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:19.279655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.280623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.281145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.281590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.282445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] RBAC: Adding query filter params (name=network) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:19.282933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.284497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.294377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 308/631] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:19.300873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:19.427736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e1d5-58bc-43dc-b55f-cffcb8868761 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.476670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb926b3-9839-4577-a907-4175e75f0794 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.491702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.492348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.492629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.492897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.493569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.494949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d81a18-72a0-47ba-9d13-52a33f7c5065 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.525840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.526464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.526818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.527097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.527741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:19.528045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.529478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.535094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 309/632] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:19.543228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:19.552831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 324/633] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:19.633541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:19.711350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b43f9-f9a8-4ac8-8f69-d9c8454064e7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:19.772758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:19.773300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:19.773576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:19.773847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:19.774530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:19.775941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23836eba-9e38-4af5-8c96-d04380151030 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:19.813576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 310/634] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:20.715938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ee68-94de-4bc5-9097-a6babebccb62 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:20.716510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ee68-94de-4bc5-9097-a6babebccb62 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:20.716510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600ee68-94de-4bc5-9097-a6babebccb62 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:20.718137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 325/635] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:20.724185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc4b8-cf6a-4136-80f4-6c7d71994a22 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:20.724722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc4b8-cf6a-4136-80f4-6c7d71994a22 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:20.724722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc4b8-cf6a-4136-80f4-6c7d71994a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:20.744237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6fdfc4b8-cf6a-4136-80f4-6c7d71994a22 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:20.745444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6fdfc4b8-cf6a-4136-80f4-6c7d71994a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:20.819444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 311/636] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:21.081686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600ee68-94de-4bc5-9097-a6babebccb62 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:21.098937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:21.159780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987ab5-700e-4afe-9312-5440b2c705d7 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.160451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987ab5-700e-4afe-9312-5440b2c705d7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.160949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987ab5-700e-4afe-9312-5440b2c705d7 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.162811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 312/637] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:21.168151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072aa1-df98-45b8-8558-c6a5a9fb626d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.168453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072aa1-df98-45b8-8558-c6a5a9fb626d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.169224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072aa1-df98-45b8-8558-c6a5a9fb626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.188891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600ee68-94de-4bc5-9097-a6babebccb62 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:21.189976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dc072aa1-df98-45b8-8558-c6a5a9fb626d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:21.191355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc072aa1-df98-45b8-8558-c6a5a9fb626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:21.223070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:21.223692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.223968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.224266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.224973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:21.225080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:21.226343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353f1e-cd87-475f-bdca-fc477d34532a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:21.244357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 326/638] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:21 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:21.290681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 313/639] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:21 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:21.392204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5f88-fd16-451a-b7d8-318fbcce4ad4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.392400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5f88-fd16-451a-b7d8-318fbcce4ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.392665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5f88-fd16-451a-b7d8-318fbcce4ad4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.394133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 327/640] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:21.400473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.401147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.401661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.421452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:21.423003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:21.498427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 314/641] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:21 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:21.568993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df5f88-fd16-451a-b7d8-318fbcce4ad4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:21.651915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:21.685779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df5f88-fd16-451a-b7d8-318fbcce4ad4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:21.719926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:21.721371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.721863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.722342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.732956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:21.738486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:21.739441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:21.740339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:21.742653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 328/642] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:15:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:21.748482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:21.775868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:21.862161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:21.872447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe9a08-96f0-4e4e-8e60-977ac10aff5a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:21.901831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fdc4a-2bff-4be0-bb02-ca53c3ba6c6e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:21.916354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:21.916895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.917172 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.917632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.918333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:21.918630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:21.920056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:21.935151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 315/643] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 11 13:15:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:21.937463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:21.951331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:21.952451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:21.952828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:21.953345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:21.954110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:21.955765 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:21.958574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:21.969641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:21.970691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:21.971850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 329/644] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:21.976839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.065229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86d84-52cd-47f7-ad56-7fdfa0b0ccaf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.084055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b95e7-e711-468f-9000-3bdae63eaf74 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.101103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.101749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.102023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.102320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.114220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.115716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.118482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.119095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.119369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.119638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.120305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:22.120609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.121972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6458d2-c53e-4d08-a02d-1933bfbe9d0e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.126625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 330/645] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:22.127648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07cfe-bf24-49a7-a4ac-e471e1e71469 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.127932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07cfe-bf24-49a7-a4ac-e471e1e71469 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.132157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07cfe-bf24-49a7-a4ac-e471e1e71469 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.132157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 331/646] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:22.132157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07cfe-bf24-49a7-a4ac-e471e1e71469 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-f62fc5a0-7fc5-497b-aafc-8f306c3103c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:22.134767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:22.135885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:22.136879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:22.136879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:22.136879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:22.136879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 316/647] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:22.137453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.137616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.137855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.155614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.162303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:22.163544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:22.199982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.244288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 317/648] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:22 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:22.244288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.249248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07cfe-bf24-49a7-a4ac-e471e1e71469 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.287942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.288682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.289023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.289354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.290046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:22.290414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.291881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.303171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 332/649] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:22.375636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76076a-6ba3-4c42-a884-1b4570d38f75 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.413139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.413770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.414049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.414321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.414981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:22.415278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.416749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.419470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.430800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 318/650] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:22.436308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.521582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.544936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c291a-d3e6-49b7-9ddf-4763bb452a6a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.578703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.593742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.594595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.595024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.595435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.605534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.607719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:22.623701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:22.624945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:22.626169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:22.628853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 333/651] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:22.635002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.644331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a071c-bff7-40b9-afa1-dfb27aac6ebf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.682463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.683109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.683440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.683720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.684342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.685841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:22.695255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:22.696305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:22.697145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 319/652] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:22.703734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.771131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364c0f6-fa11-4850-9f0d-253b96adc4ea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.779160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.820723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.821753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.822036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.822344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.823102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:22.823399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.824942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.833354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f227-09bf-44cd-8ad0-38586dcab8fa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.848475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 334/653] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:22.850290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.868604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:22.869216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:22.869489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:22.869757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:22.870433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:22.870731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:22.872159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:22.877553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 320/654] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:22.883143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:22.916865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:22.990869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193dd0a-fe42-45e4-a2eb-033f1410f502 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:22.991422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308ead-a183-49d8-bc3c-8b542468e7b9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:23.015304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:23.028317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.028831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.029107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.029426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.030106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:23.030416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.031759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.041487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.042505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.043004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.043470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.044269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 321/655] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:23.056446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.058312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:23.076580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:23.077590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:23.078489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:23.078489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:23.078489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:23.080753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 335/656] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:23.086772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:23.200096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fda0-7ab7-4488-8319-aabe214d1f9f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:23.235928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.236996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.237274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.237543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.238219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:23.238525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.239926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.254178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 322/657] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:23.259710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:23.362360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:23.369567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c076bc-d6a5-46fd-ae15-de45415adefc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:23.405209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.405877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.406150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.406418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.414641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.416235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-28b453ec-2e1b-4b4d-91ba-26aefbdec8ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:23.429360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:23.430338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:23.431217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 336/658] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:23.436776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:23.488358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:23.526341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31247f75-0371-4837-adb5-30d817856274 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.526536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31247f75-0371-4837-adb5-30d817856274 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.526785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31247f75-0371-4837-adb5-30d817856274 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.529461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 337/659] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:23.533364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.533879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.533879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.543564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98b6f-48d1-441e-aeff-48c30450662a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:23.555946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:23.557446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:23.594020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.594646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.594921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.595188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.595891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:23.596220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.597644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.617555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 323/660] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:23.618517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:23.660260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 338/661] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:23.723290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d069b-41f6-410c-84ab-2a71243fc942 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:23.732188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:23.757132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:23.757801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:23.758080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:23.758354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/70eff2211f2741b1b3fc3f9d0c660d60/roles/db53cf6b98df4bb492c10be037c05ac5` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:23.760608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=70eff2211f2741b1b3fc3f9d0c660d60, role_id=db53cf6b98df4bb492c10be037c05ac5)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:23.762039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:23.777537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 324/662] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:23 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/70eff2211f2741b1b3fc3f9d0c660d60/roles/db53cf6b98df4bb492c10be037c05ac5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:15:23.935495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:24.024686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:24.047638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c3fe-202d-4cb8-8b47-d211b883a126 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:24.081791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:24.082775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.083265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.083747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.084571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:24.086133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:24.095156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:24.096085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:24.097122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:24.104338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 339/663] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 11 13:15:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:24.105886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:24.213308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a943b-c519-42d3-9460-66181775c33c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:24.247579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:24.248229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.248511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.248787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.249428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:24.249732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:24.251108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:24.256347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 325/664] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:24.264953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:24.366605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d5f3a-78a2-4790-8d6f-1946c294817f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:24.402407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:24.403103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.403383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.403653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.404351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:24.404656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:24.406087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f649db-6a8b-435b-8103-fa227323d47a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:24.412398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 340/665] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:24.418642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:24.476593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:24.521376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77242-1d7b-4f9c-8b73-308f59965e55 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.521777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77242-1d7b-4f9c-8b73-308f59965e55 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.522461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b77242-1d7b-4f9c-8b73-308f59965e55 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.523811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 341/666] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:24.530669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.530953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.531087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec05e-d849-420a-930f-6a4ba33f1542 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:24.531270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.558253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:24.560311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:24.565452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:24.566073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.566347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.566615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.567319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] RBAC: Adding query filter params (interface=public, service_id=4ea8a66d1f0645abb92c204b64748b12, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:24.573124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:24.574534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:24.580002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 326/667] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:24 2022] GET /identity/v3/endpoints?service_id=4ea8a66d1f0645abb92c204b64748b12&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:24.581451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:24.591498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:24.648150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 342/668] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:24 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:24.699312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:24.705835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d12e59-da11-4cc1-908f-e4d0382e3a4c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:24.741509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:24.742105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:24.742389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:24.742656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:24.743275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:24.744736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194b0d-f8b8-4cff-87de-8ca7cc7c2644 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:24.752641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 327/669] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:24 2022] GET /identity/v3/services => generated 1029 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:15:24.915242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:25.019069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761b6d-6c64-4939-b2aa-aaf2d3b75fc8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:25.054482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:25.055099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:25.055432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:25.055699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:25.056444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:25.057819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301e5d6-8000-42ac-a6e3-6f4e42f1b25d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:25.075515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 343/670] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:24 2022] POST /identity/v3/services => generated 241 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:25.303966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e558f-fd39-463f-868c-131a985b5332 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:25.304480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e558f-fd39-463f-868c-131a985b5332 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:25.304480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e558f-fd39-463f-868c-131a985b5332 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:25.306042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 328/671] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:25.311334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95fa7-9e28-4b2b-befd-bb3ceda9ddd6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:25.311618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95fa7-9e28-4b2b-befd-bb3ceda9ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:25.311889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95fa7-9e28-4b2b-befd-bb3ceda9ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:25.319298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:25.331089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cff95fa7-9e28-4b2b-befd-bb3ceda9ddd6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:25.332291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cff95fa7-9e28-4b2b-befd-bb3ceda9ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:25.409502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 344/672] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:25 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:25.678992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e558f-fd39-463f-868c-131a985b5332 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:25.785397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e558f-fd39-463f-868c-131a985b5332 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:25.820510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:25.821167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:25.821446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:25.821716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:25.822377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:25.823743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012ebd-680c-4204-be72-d2803b1c4cb1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:25.841048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 329/673] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:25 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:26.045533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384481-f6d2-4dee-b774-9aa2ff779431 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.046496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384481-f6d2-4dee-b774-9aa2ff779431 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.046776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384481-f6d2-4dee-b774-9aa2ff779431 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.048320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 345/674] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:26.054226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.054783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.055255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.074178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:26.075678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:26.091766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:26.153702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 330/675] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:26.419129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77384481-f6d2-4dee-b774-9aa2ff779431 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:26.521580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77384481-f6d2-4dee-b774-9aa2ff779431 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:26.549987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:26.556052 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:26.557054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.557681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.557954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.568642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:26.573442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:26.574431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:26.575297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 346/676] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:15:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:26.582433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:26.693659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47fe0c-15a3-4fad-b840-4f189f19ec54 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:26.728030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:26.728676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.728958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.729230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.729904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:26.730211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:26.731590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:26.740299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 331/677] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 11 13:15:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:26.746064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:26.859095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdf91-0ef0-4fef-bac6-ea61c13fe4fd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:26.894699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:26.895678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.896177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.896694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.907632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:26.909415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e6bb1512-ce5d-4666-abf1-07cbf1378b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:26.925305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:26.926294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:26.927178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:26.927178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:26.927178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:26.929402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 347/678] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:26.935282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:26.941361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b5cd9-5e06-45f3-b6fd-7bbc3ffb0365 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.941641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b5cd9-5e06-45f3-b6fd-7bbc3ffb0365 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.941909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b5cd9-5e06-45f3-b6fd-7bbc3ffb0365 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.943553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 348/679] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:26.948551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:26.948781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:26.949050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:26.949379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:26.974929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:26.976722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:27.019058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.053546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d719ac3-c010-4230-9fd8-c7aaaa2d9c3d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.064199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 349/680] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:27.091424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.092451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.092964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.093429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.094276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:27.094782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.096441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.109227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 332/681] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:27.115194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.222253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84acd3e-5311-42b9-a297-3e8c43d5bd1b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.256480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.257204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.257484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.257753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.265140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.266674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-296846d4-2327-4b8e-ba00-4352c4a2b613 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:27.279252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:27.280309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.281266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 350/682] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:27.287805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.308968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2ce6-d8da-4013-b9ae-4225b98b8510 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.309287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2ce6-d8da-4013-b9ae-4225b98b8510 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.309510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2ce6-d8da-4013-b9ae-4225b98b8510 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.311113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 351/683] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:27.315904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.316212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.316490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.334629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:27.335808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:27.362086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.363380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.399095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbe6f5-62d5-48a6-a562-ded447c90ec1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.406094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 352/684] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:27.413304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.434892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.435602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.435955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.436406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.437517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:27.437625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.439137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.462476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 333/685] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:27.463526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.529815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423df8-7d76-4d78-a425-82027c4b0f9f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.565858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.566798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.567768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.568265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.569099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.570654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:27.582210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:27.583147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 13:15:27.584026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.587186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 353/686] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:27.592724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345256f4-76ad-46f3-9bab-bec231a0ef29 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.593192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.621676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.627097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.638683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.639524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.639943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.640392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.707852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391e30c-2918-4054-8d7b-617936d39e6f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.717854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.719302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.742724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.743333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.743606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.743868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.744552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:27.744942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.746314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.751006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 354/687] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:27.752270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.837650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 334/688] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:27.837650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.845442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.855555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41648550-abba-4dd2-8e3f-b7701c8c3693 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.864361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:27.888633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.889268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.889546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.889811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.890419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.891786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:27.900616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:27.901581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:27.902434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 355/689] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:27.903288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:27.947560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975fd4f-ac6d-40c3-aa25-36ee14399ce2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:27.985783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:27.986696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:27.987135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:27.987388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:27.988015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:27.988343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:27.989690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:27.999311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 335/690] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:27.999311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.011858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d09104-e6a9-488c-88a3-0b8ca71934ef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.049005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.049671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.049955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.050225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.062084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.063564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:28.078824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:28.080615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:28.081712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:28.081712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:28.081712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:28.081712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 356/691] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:28.081712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.090309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.102278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.112946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2bb73-0de2-4a0e-a9f1-2f422c787f0d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.148757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.149365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.149636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.149926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.150520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.150818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.152202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.157306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 336/692] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.158144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.188955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2cf23-1aa8-4715-a813-2452780e212e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.222470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.223090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.223369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.223641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.224333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.225724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80b8c6-1df0-44f6-afaf-1cada2053f87 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.245289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 357/693] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:28.246597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16464a5-6805-40ab-855c-e34a71347870 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.246885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16464a5-6805-40ab-855c-e34a71347870 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.247157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16464a5-6805-40ab-855c-e34a71347870 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.252365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 358/694] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:28.252365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16464a5-6805-40ab-855c-e34a71347870 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.261888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911e58-2293-45f2-a0e7-0d50f5d02f8b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.309297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.309929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.310206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.310476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.311211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.311655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.313120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f859bf5-f855-4cbc-a397-042010761b38 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.332013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 337/695] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.332013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.332013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.332013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.333304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.352417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:28.353978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:28.358046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.378825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16464a5-6805-40ab-855c-e34a71347870 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.416649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.417247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.417997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.418229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.418872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.419185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.420628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.427525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 359/696] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.427665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.433379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 338/697] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:28.434251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.548216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7a9a-81f4-4b01-9777-7fd7823b2523 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.548873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85aaedd-8ee5-4692-9021-2852d2830518 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.579427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.589990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.590479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.590752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.591018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.596401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.598551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.599032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.599136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.599623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.600660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.600788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] RBAC: Adding query filter params (interface=public, service_id=6280e7a0d27b4c278c5b83434f4e5420, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.601463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.603759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.610131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 339/698] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/endpoints?service_id=6280e7a0d27b4c278c5b83434f4e5420&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:28.616258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:28.617180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:28.618064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 360/699] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:28.618577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.619566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.632926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.722966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8a195-be51-4ffe-948c-dc670db88daa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.739170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191a5c-92f3-4314-9ced-e5e12a5a5688 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.757684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.758298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.758571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.758840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.759530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.760068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.761299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.774262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 361/700] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.774877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.775228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.775814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.777803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.777803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.777803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.780026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.788629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 340/701] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:15:28.788629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:28.849265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.874612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:28.912630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216cc5-9fde-44a6-b6b3-5698945f153e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.931592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c93aae-66e1-4197-be07-d1cd55be1fa8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:28.950436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.951155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.951503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.951843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.952791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] RBAC: Adding query filter params (role.id=db53cf6b98df4bb492c10be037c05ac5, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=70eff2211f2741b1b3fc3f9d0c660d60) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:28.953086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.954598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.961476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 341/702] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/role_assignments?user.id=70eff2211f2741b1b3fc3f9d0c660d60&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=db53cf6b98df4bb492c10be037c05ac5 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:28.976732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:28.977697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:28.978137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:28.978401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:28.979004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:28.980464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:28.990516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:28.991588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:28.992822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 362/703] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:29.000911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:29.120273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e4094-9a66-49b3-ad54-8f4ff847ea8d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:29.152438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:29.159558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:29.160284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:29.160485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:29.160756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:29.161397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:29.161713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:29.163142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:29.168560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 342/704] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:29.174495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:29.284856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ff62c-ca53-46c1-96a6-3ff290c95386 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:29.299952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:29.324818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:29.325442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:29.325719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:29.325989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:29.326628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:29.326930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:29.328352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:29.334588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 363/705] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:29.340516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:29.444561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f0ffe-0f0d-4b87-9ece-e1d2da0ec9f2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:29.479669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:29.480658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:29.481128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:29.481637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:29.482533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] RBAC: Adding query filter params (interface=public, service_id=a51321c5549b4db29408c9d60bae2c5c, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:29.483030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:29.484641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c5aa0-a84c-476e-886c-22abfb175d66 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:29.495412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 343/706] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:29 2022] GET /identity/v3/endpoints?service_id=a51321c5549b4db29408c9d60bae2c5c&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:29.497200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:29.608843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7be61-1c68-407e-8d98-391a1516e772 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:29.645588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:29.646256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:29.646561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:29.647223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:29.647829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:29.649233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1a02c-f1c0-4fec-aac3-ca6100ce893b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:29.659828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 364/707] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:29 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:15:30.528014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a515a-ae37-4d2f-92e6-9c9c075d419a None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:30.529064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a515a-ae37-4d2f-92e6-9c9c075d419a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:30.529540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a515a-ae37-4d2f-92e6-9c9c075d419a None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:30.531489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 344/708] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:30.537170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9df50c-f9b4-4bc4-8c37-0a9d00ae7048 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:30.537721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9df50c-f9b4-4bc4-8c37-0a9d00ae7048 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:30.537798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9df50c-f9b4-4bc4-8c37-0a9d00ae7048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:30.557228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cb9df50c-f9b4-4bc4-8c37-0a9d00ae7048 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:30.558475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cb9df50c-f9b4-4bc4-8c37-0a9d00ae7048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:30.596597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:30.637530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 365/709] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:30.900967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a515a-ae37-4d2f-92e6-9c9c075d419a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:30.962431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.004132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a515a-ae37-4d2f-92e6-9c9c075d419a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.004955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19764b-3275-4fd7-9fce-0536278e6efa None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.005240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19764b-3275-4fd7-9fce-0536278e6efa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.005510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19764b-3275-4fd7-9fce-0536278e6efa None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.006913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 366/710] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:31.012160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31d18-64ce-4f3a-88d5-e3eaa8f5c9a7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.012467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31d18-64ce-4f3a-88d5-e3eaa8f5c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.012743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31d18-64ce-4f3a-88d5-e3eaa8f5c9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.035652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e3d31d18-64ce-4f3a-88d5-e3eaa8f5c9a7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:31.036857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e3d31d18-64ce-4f3a-88d5-e3eaa8f5c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:31.039407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.040087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.040389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.040661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.041246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.042657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:31.051827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:31.053050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.053909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 345/711] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:31.059105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.128356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 367/712] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:31.167606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888486-c947-46a3-a10d-19845092c1a3 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.191344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.192643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37c64d-0ef7-424a-8e18-0172c388aae4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.192953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37c64d-0ef7-424a-8e18-0172c388aae4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.193288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37c64d-0ef7-424a-8e18-0172c388aae4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.198786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 368/713] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:31.200022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.200414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.200605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.210395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.211006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.211281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.211550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.212209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:31.212510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.213956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.219084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 346/714] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:31.220005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:31.221219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:31.224826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.296746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 369/715] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:31.327831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3038-6bf2-4495-b730-1574b94220c4 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.361823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.362440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.362716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.362984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.363616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:31.363912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.365341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.370085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 347/716] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:31.413285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.444429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.524833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67bbf4-f06c-4bee-baac-5168d9f47c6a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.565911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.566585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.566854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.567123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.567713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.569213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.570113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:31.578796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:31.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:31.582164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 370/717] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:31.587961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.648593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.707284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563e53-3365-437c-bb26-41288b653cf6 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.712979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.728388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7c1f0-e332-4a59-9dca-7085996d868b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.752472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.753947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.754657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.755136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] PATH_INFO: `/v3/services/image` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.756178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.757889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.769166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154941d9-120c-4229-a0af-35268145b03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:31.769528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:31.770492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:31.771350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:31.772518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154941d9-120c-4229-a0af-35268145b03a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.772518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154941d9-120c-4229-a0af-35268145b03a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.772518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154941d9-120c-4229-a0af-35268145b03a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.772518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154941d9-120c-4229-a0af-35268145b03a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:31.772518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154941d9-120c-4229-a0af-35268145b03a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.773731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154941d9-120c-4229-a0af-35268145b03a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.780477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 348/718] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services/image => generated 86 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:31.781089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.786466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 371/719] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:31.792138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154941d9-120c-4229-a0af-35268145b03a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.909506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90fa3-e5d1-4e42-a024-bbeaa71e5212 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.914486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154941d9-120c-4229-a0af-35268145b03a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:31.931355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:31.949118 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.949734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.950010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.950295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.950903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] RBAC: Adding query filter params (name=image) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:31.951198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.951532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:31.952225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.952506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.952598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d885b-d9a3-485a-93c1-04b13d290629 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.952775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.953444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:31.953744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:31.955109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc7aa-8be0-4474-9a3d-f453ab4bd3e2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:31.958442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 349/720] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:31.959945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbdbd-53c6-4002-8769-383235eed4a9 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.960426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbdbd-53c6-4002-8769-383235eed4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.960893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbbdbd-53c6-4002-8769-383235eed4a9 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.962481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 350/721] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:31.963537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbbdbd-53c6-4002-8769-383235eed4a9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:31.968163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 372/722] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:31.968163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:31.968163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:31.968427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:31.986428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:31.987609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:32.002176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:32.059648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 373/723] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:32.060288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:32.077544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbbdbd-53c6-4002-8769-383235eed4a9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.122107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.123444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.123930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.124466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.125901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:32.126404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.128682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.140146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 351/724] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:32.195368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244e6b8-a21c-4305-aa55-72195ae9658c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.234045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.234611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.234877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.235148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.235829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.237257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.262029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 374/725] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:32.331849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:32.385195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:32.446178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7a17-b711-4717-835f-92f1fe68f4d8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.485837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.486373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.486650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.486920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.487511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.488987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:32.498293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:32.499305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:32.500426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 352/726] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:32.507735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:32.519525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:32.620447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6063696-03c4-4046-8070-33fe4d6f3ce7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.655326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.655948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.656251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.656520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.657157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:32.657459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.658832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.664058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 375/727] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:32.669698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:32.777546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac065273-0d68-412a-b097-a1557ca9d8a9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.779890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:32.814774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.815389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.820145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.820145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.820145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:32.820145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.820145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138169-8796-4579-86fb-34b53b3f6d85 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.823794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 353/728] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 11 13:15:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:32.829365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:32.899409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:32.934993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e284c9-fca5-4813-a0a5-d7e53dd90bf0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:32.969170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:32.969785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:32.970061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:32.970330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:32.971005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:32.972400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf00e3-0612-412a-8a8b-a4667c826a4b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:32.999045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 376/729] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:33.593206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ede59-d1be-497d-a2b1-7a0d7340efc0 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:33.593631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ede59-d1be-497d-a2b1-7a0d7340efc0 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:33.594012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ede59-d1be-497d-a2b1-7a0d7340efc0 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:33.595722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 354/730] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:33.600901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27af01-a900-45a6-850e-f60d8a748432 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:33.601187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27af01-a900-45a6-850e-f60d8a748432 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:33.601462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27af01-a900-45a6-850e-f60d8a748432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:33.619770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7f27af01-a900-45a6-850e-f60d8a748432 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:33.620971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7f27af01-a900-45a6-850e-f60d8a748432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:33.694739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 377/731] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:33.956788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ede59-d1be-497d-a2b1-7a0d7340efc0 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:34.064658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ede59-d1be-497d-a2b1-7a0d7340efc0 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:34.104620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:34.105237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.105513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.105785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.106445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:34.107806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cda0b6-c49b-4f53-b394-a11e9a05a5f9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:34.126381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 355/732] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:33 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:34.463520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5517-7aec-481d-83c4-6a350e1ac785 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.463945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5517-7aec-481d-83c4-6a350e1ac785 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.464135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5517-7aec-481d-83c4-6a350e1ac785 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.465719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 378/733] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:34.471079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.471370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.471642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.488667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a663-83eb-4986-85ba-df6375224299 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.489268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a663-83eb-4986-85ba-df6375224299 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.489737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a663-83eb-4986-85ba-df6375224299 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.490801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:34.491822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 379/734] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:34.492001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:34.497244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.497764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.498224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.516302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:34.517778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:34.518525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:34.580497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 356/735] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:34.584635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:34.604745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 380/736] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:34.848634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:34.868460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:34.957902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71169b6a-e516-4caa-bb75-b538bfbe6a61 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:34.971253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7da2e-f3d1-40e1-a7c1-370e09980ec5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:34.997143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:34.998066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:34.998357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:34.998630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:34.999717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.001418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f7029-5e47-455d-b000-062c3b8a603f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.008418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.009534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.009534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.010004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.010699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.012515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:35.025195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:35.026286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:35.027184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:35.036340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 381/738] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:35.036340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.036890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 357/738] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:34 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:35.088662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:35.156994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c43e42-b18e-4499-a398-efb0d49e22e4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.200902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200217-4651-4fc9-b76a-6934028bc52f None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.201231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200217-4651-4fc9-b76a-6934028bc52f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.201450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200217-4651-4fc9-b76a-6934028bc52f None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.203072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.203964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.204499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.204624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 358/739] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:35.205136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.206080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:35.206650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.208315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.209165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.209447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.209719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.214897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 382/740] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:35.220945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.235772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:35.237045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:35.252065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:35.336308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 359/741] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:35 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:35.349995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcb8c7-18ed-4f2b-a332-a043b5025799 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.357458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:35.386180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.386802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.387127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.387404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.388098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:35.388458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.389892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.395231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 383/742] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:35.401571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.500158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1421d7-2e32-4e0f-9709-52765b0daeac None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.534656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.535593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.536064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.536549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.537508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] RBAC: Adding query filter params (interface=admin, service_id=6280e7a0d27b4c278c5b83434f4e5420, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:35.538010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.539547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.545270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 360/743] 10.222.0.15 () {56 vars in 1249 bytes} [Tue Oct 11 13:15:35 2022] GET /identity/v3/endpoints?service_id=6280e7a0d27b4c278c5b83434f4e5420&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:35.550796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.624151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.639600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:35.674047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97904f-4e3e-4794-887b-1e035ddad644 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.719011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.719986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.720689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.721575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.722663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.724555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.740661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 384/744] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:35 2022] GET /identity/v3/services => generated 1957 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:15:35.747803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8416-93d4-481d-ade5-4268229b18d9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.789940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.790490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.790767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.791036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:35.791691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:35.793150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:35.795922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING keystone.server.flask.common [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 11 13:15:35.817972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.sql.core [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:35.817972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:15:35.817972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [parameters: {'id': 'b40a0a5869084f129d1f59026ca19fa2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 13:15:35.817972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81370) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b40a0a5869084f129d1f59026ca19fa2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b40a0a5869084f129d1f59026ca19fa2. Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:35.826459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 13:15:35.827343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application yield resource Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise self.value Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application session.commit() Oct 11 13:15:35.828342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.flush() Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application compat.raise_( Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 13:15:35.829246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 13:15:35.830160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:35.830964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [parameters: {'id': 'b40a0a5869084f129d1f59026ca19fa2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.831843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:35.834477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b40a0a5869084f129d1f59026ca19fa2. Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:35.836665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 361/745] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 11 13:15:35 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:35.844934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:35.884133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:35.957876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0020c3c-5491-48df-847d-cf2d23bac54e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:35.993734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:35.994725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:35.995227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:35.995693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.002952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:36.004683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9d8337df-1265-40ba-8fb1-5c1c150754f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:36.016731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:36.017668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:36.018562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:36.020645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 385/746] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:36.027287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:36.099560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:36.138197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1470630-c47d-4fa5-af6a-dc7b26b9fdcf None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:36.175062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:36.175746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:36.176025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:36.176325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.177019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:36.177328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:36.179063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a7beb-0f81-4337-b228-6cedd79a3552 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:36.202168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 362/747] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:36.289831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590311-948a-442c-85f4-c088ea3e2ea9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:36.290165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590311-948a-442c-85f4-c088ea3e2ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:36.290371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590311-948a-442c-85f4-c088ea3e2ea9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.292021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 386/748] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:36.297880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:36.298225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:36.298433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.319948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:36.321550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:36.395109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 363/749] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:36 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:36.656060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590311-948a-442c-85f4-c088ea3e2ea9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:36.755662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:36.766994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590311-948a-442c-85f4-c088ea3e2ea9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:36.802146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:36.802821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:36.803097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:36.803418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.804003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:36.805521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:36.814515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:36.815553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:36.816429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 387/750] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:36.823856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:36.880909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:36.926322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa0b43-4769-4d04-a1d2-d450d6b55b0a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:36.961014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:36.961987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:36.962478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:36.962967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:36.963796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:36.964324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:36.965880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:36.971546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 364/751] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:36.977513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:37.082979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612ff6d-0dc4-4f96-bc65-a8cd0123e499 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:37.118800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:37.119794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.120298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.120789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.121612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:37.122142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:37.123704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:37.129317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 388/752] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:37.135578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:37.244487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7a20-b8c5-4a09-8b32-79d65442a6f9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:37.267281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:37.285741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:37.286366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.286645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.286918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.287650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] RBAC: Adding query filter params (interface=public, service_id=8a8a3cb67ef24a7285b899b88632616a, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:37.287960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:37.289360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1c42e-e760-4846-9ae0-7d5cbf2595e6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:37.294808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 365/753] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:37 2022] GET /identity/v3/endpoints?service_id=8a8a3cb67ef24a7285b899b88632616a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:37.300522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:37.326107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:37.330236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27bf57-0c99-4e7e-8b81-e5d8c4bc4bd9 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.330697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27bf57-0c99-4e7e-8b81-e5d8c4bc4bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.330972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27bf57-0c99-4e7e-8b81-e5d8c4bc4bd9 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.333282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 366/754] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:37.340385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.340788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.341163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.364109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:37.365992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:37.401932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028d04-1793-452a-9bf1-022113ad6b63 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:37.436496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:37.437165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.437443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.437717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.438372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:37.439752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:37.448038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 367/756] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:37.452216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 389/756] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:37 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:15:37.713575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:37.814282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0ab32-1503-4951-b853-68af1aaa409e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:37.829286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:37.855765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:37.856369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.856651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.856919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] PATH_INFO: `/v3/services/image` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.857511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:37.858895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-b8cd50ec-253d-498b-a553-afe71bcaf563 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:37.868631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:37.869645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:37.870687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 368/757] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 11 13:15:37 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:37.877843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:37.949823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e767ad-b0a6-4253-b2a3-4b741749984c None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.950130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e767ad-b0a6-4253-b2a3-4b741749984c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.950379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e767ad-b0a6-4253-b2a3-4b741749984c None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.951742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 369/758] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:37.957389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:37.957673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:37.957952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:37.966526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:37.979009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:37.980898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:37.982924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205a24f-172a-4313-a822-65ded3e4f04b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.019225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.020351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.020923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.021470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.022683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:38.023195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.024852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.030729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 390/759] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:38.036353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.060407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 370/760] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:38.149100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cf96e-2eaf-47af-adaa-56636ab37db8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.183562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.184204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.184480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.184743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.185378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:38.185674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.187044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.192415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 391/761] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:38.198259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.300563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf690539-5df5-42c2-a969-ed85fd3bf4a2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.312528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:38.337055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.337806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.338145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.338662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.339468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] RBAC: Adding query filter params (interface=public, service_id=7c5b193b68b44a4d8f0a9a3f14fc6908, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:38.339842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.341347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.352231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 371/762] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/endpoints?service_id=7c5b193b68b44a4d8f0a9a3f14fc6908&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:38.353841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.382689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:38.383595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.462234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb704af-60ad-4bb0-be95-1fbe1a8541b1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.491754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84150294-fc55-455e-9772-81d9305b78f2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.496571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.497448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.497921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.498384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.499184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.500826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.508074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 392/763] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:15:38.535468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.536588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.537114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.537707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.538541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.540248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-94b23fac-9b2f-4c85-aab1-50d1850019b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:38.550421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:38.551520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 13:15:38.552396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:38.555472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 372/764] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:38.561807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.648166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:38.654322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61987a-7b55-413c-9c13-21d58339758e None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.655644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61987a-7b55-413c-9c13-21d58339758e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.655644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61987a-7b55-413c-9c13-21d58339758e None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.657415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 373/765] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:38.663287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.663864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.664399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.673509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab4358-5380-4f5a-8194-1a06ff7fcf24 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.684422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:38.685989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:38.710522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.711161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.711404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.711673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.712392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:38.712637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.714018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.720465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 393/766] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:38.726545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.728327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:38.788756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 374/767] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:38.840624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fc80-7dc9-421f-9b84-3be5091b9770 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:38.875205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:38.875821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:38.876093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:38.876384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:38.877008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:38.877365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:38.878790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:38.884325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 394/768] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 11 13:15:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:38.889728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:38.987144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5001ef-63ee-4974-abda-9ed63de0522c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:39.020904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446d468-b460-4218-9088-08745b9e373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:39.022234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d468-b460-4218-9088-08745b9e373e None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.022711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d468-b460-4218-9088-08745b9e373e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.023177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d468-b460-4218-9088-08745b9e373e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.024065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446d468-b460-4218-9088-08745b9e373e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:39.025642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446d468-b460-4218-9088-08745b9e373e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:39.047961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 375/769] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:39.098015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:39.109131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:39.215966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6693c-dc8b-4762-be29-269841b3d589 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:39.258560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:39.259567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.260043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.260546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.261377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:39.263020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:39.265897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING keystone.server.flask.common [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 11 13:15:39.280426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 13:15:39.280426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 13:15:39.280426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [parameters: {'user_id': '0fe1d66d786f4ae691f0abe68f8c1380', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 13:15:39.280426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:39.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:39.289913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 13:15:39.290886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 13:15:39.291791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:39.292711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:39.293541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'user_id': '0fe1d66d786f4ae691f0abe68f8c1380', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:39.294359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 11 13:15:39.295139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:39.296163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 395/770] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 11 13:15:39 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:39.300902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446d468-b460-4218-9088-08745b9e373e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:39.349372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:39.410452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446d468-b460-4218-9088-08745b9e373e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:39.445564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:39.446565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.447069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.447542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.455972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:39.457711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8c035968-d211-475b-bbaa-0b9ce0b3d187 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:39.469650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:39.470614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 13:15:39.471556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:39.474159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 376/771] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:39.480625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:39.557452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:39.610536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20894-4c68-455a-8e8d-b23c7668bedc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:39.643630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1aca1-2d80-4cd3-b781-83c54710e771 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.643950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1aca1-2d80-4cd3-b781-83c54710e771 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.644212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1aca1-2d80-4cd3-b781-83c54710e771 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.644378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:39.645049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.645328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.645602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.645833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 377/772] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:39.646352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:39.648099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:39.650047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:39.652231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:39.652231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:39.652231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:39.670445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:39.671642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:39.700227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 396/773] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 11 13:15:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:39.748107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 378/774] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:39 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:40.009458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:40.111838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772321-f113-47d6-bb5b-2bd13cfc2401 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:40.134899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:40.148425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:40.149070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.149345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.149664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.150247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:40.151611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:40.160473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:40.161441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 13:15:40.162288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:40.164205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 397/775] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:40.167579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:40.183485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:40.272381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aebb6-5f1d-4055-94b7-096745531759 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:40.307114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:40.307745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.307962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.308290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.308925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:40.309227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:40.310738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:40.320150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 379/776] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:40.322037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:40.426994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df313b-ff2b-4188-972b-88ddd342ffa7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:40.460548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:40.461161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.461437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.461706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.462349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:40.462636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:40.464030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f20ad3-7542-408c-9f43-ac0053474eff None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:40.469078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 398/777] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 11 13:15:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:40.474725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:40.552445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:40.578787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569e8cf-0f1c-44da-b323-ada665651c91 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:40.615682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:40.616325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.616603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.616869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.617692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:40.619045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2ba32-f80d-4c01-b161-18bc882114ea None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:40.648165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 380/778] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:40.771542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395b72-8553-454b-868f-705d480806ed None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.772220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395b72-8553-454b-868f-705d480806ed None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.772521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395b72-8553-454b-868f-705d480806ed None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.773941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 399/779] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:40.779894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d1565-9d56-46a2-8db0-275dc1f6bdd7 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:40.780292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d1565-9d56-46a2-8db0-275dc1f6bdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:40.780481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d1565-9d56-46a2-8db0-275dc1f6bdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:40.799762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-482d1565-9d56-46a2-8db0-275dc1f6bdd7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:40.800993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-482d1565-9d56-46a2-8db0-275dc1f6bdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:40.880498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 381/780] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:41.143103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37395b72-8553-454b-868f-705d480806ed None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.199079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:41.252633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37395b72-8553-454b-868f-705d480806ed None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.288017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.288691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.288963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.289227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] PATH_INFO: `/v3/services/image` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.289802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.291261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:41.292788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52e816-cd2b-43b5-9d0f-bf9dc2f69852 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.293279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52e816-cd2b-43b5-9d0f-bf9dc2f69852 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.293559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52e816-cd2b-43b5-9d0f-bf9dc2f69852 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.295626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 382/781] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:41.300527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:41.301754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 13:15:41.302836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:41.304108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.304423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.304692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.306430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 400/782] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 11 13:15:41 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:41.307424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.327822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:41.329053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:41.392404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:41.407748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 383/783] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:41.422507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920734a-e8b6-4e45-8f5c-54bb7c54204c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.437802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:41.460269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.461255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.461787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.462257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.463104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:41.463615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.465294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:41.470840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 401/784] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:41.476617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.584416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c891dc0-219c-46f0-8a88-3a2b15592a1c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.625195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.625809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.626077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.626342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.626971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] RBAC: Adding query filter params (type=image) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:41.627266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.628657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:41.633679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 384/785] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:41.639408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.668282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.725151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:41.740918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52bf59-3639-4d2e-80e1-fe342ccaced7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.772309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab8735-a6ef-4240-ab22-ba5cf1b192dc None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.773428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.774035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.774307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.774571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.775238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.776630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:41.798367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 402/786] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:15:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:41.816693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.817410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.817686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.817952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.819201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.820740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47eab0-16e2-4f93-a87f-aa606c2aff93 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:41.822621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 385/787] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:41.828544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:41.866355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:41.941157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673b63-0355-466a-89f6-dc58a93ae7e2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:41.977720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:41.978370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:41.978648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:41.978917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:41.979572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:41.981071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627c91-bc80-4eb6-bd3f-2f635cc16017 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:42.009728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 403/788] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:15:41 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:42.878960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61666-17f8-4ddb-8ae3-cfe9f6e288b4 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:42.879489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61666-17f8-4ddb-8ae3-cfe9f6e288b4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:42.879489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b61666-17f8-4ddb-8ae3-cfe9f6e288b4 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:42.881009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 386/789] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:42.886031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:42.886449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:42.886525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:42.906690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:42.908308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:42.977395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 404/790] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:43.238507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61666-17f8-4ddb-8ae3-cfe9f6e288b4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:43.341633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b61666-17f8-4ddb-8ae3-cfe9f6e288b4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:43.358967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:43.376547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:43.377180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.377456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.377723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.387861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:43.392649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:43.393578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:43.394433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:43.395680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 387/791] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:15:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:43.400041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:43.417532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:43.501376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938873cd-7841-4326-b66e-25d6d8e66748 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:43.534962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:43.535577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.535852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.536143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.536870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:43.537183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:43.538564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:43.546821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 405/792] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 11 13:15:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:43.552464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:43.662037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f44f26-f0f3-401f-9287-4b4aeae04c32 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:43.701258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:43.701885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.702168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.702420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.712687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:43.714108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:43.728600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:43.729541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:43.735359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:43.735359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:43.735359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:43.735359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 388/793] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:43.735589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:43.815838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:43.841081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a7be-1adf-4e67-b473-88ce019d6386 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:43.876434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:43.877045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.877314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.877574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.878201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:43.878495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:43.879859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1065c9-131f-42d4-80ad-b16c3ec235aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:43.891802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 406/794] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:43.897690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:43.928525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:43.948424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d466d-2ce0-4828-be83-686925f4e848 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.948976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d466d-2ce0-4828-be83-686925f4e848 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.949245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d466d-2ce0-4828-be83-686925f4e848 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.950593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 407/795] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:43.955732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:43.955987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:43.956280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:43.975304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:43.976817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:44.001934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55249f-5d78-4d62-9af8-489804fc4b2f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.035772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.036438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.036713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.037008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.045022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.046482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.048446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 408/796] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2874c81c-7b78-4ba2-8438-3b8f20ecdd63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:44.062999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:44.064027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:44.065653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.066595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 389/797] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:15:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:44.070554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.138789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.173341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eecb6-e649-42e9-9045-856c2d309c89 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.173669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eecb6-e649-42e9-9045-856c2d309c89 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.173882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eecb6-e649-42e9-9045-856c2d309c89 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.174674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f937eba-b1a5-4ca8-9108-96f1cecf9592 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.175392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 390/798] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:15:44.180198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.180482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.180751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.198635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:44.199811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:44.210559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.211173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.211564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.211836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.212594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:44.212893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.214299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.231084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 409/799] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:44.236826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.273495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 391/800] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:15:44.315195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.338334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.342405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403274ee-a5c8-403a-9fa4-526e03c4d773 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.368598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.377293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.377964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.378263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.378515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.388544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.389996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:44.404912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:44.405859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:44.406747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:44.406747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:44.406747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:44.406747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 410/801] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:44.411624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.428990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5a86b-3a42-43ff-b6a6-0e6985273478 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.469339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.470225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.470679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.471278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.472514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.474150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.484632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.sql.core [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:44.484632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:15:44.484632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [parameters: {'id': 'd4092cc3a5454d64a56e171c8775997b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 13:15:44.484632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81370) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:44.491989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 13:15:44.492850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application yield resource Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application next(self.gen) Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise self.value Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application session.commit() Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 13:15:44.494604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.session.flush() Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application compat.raise_( Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 13:15:44.495579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 13:15:44.496527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application conn.query(q) Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application result.read() Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 13:15:44.497283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [parameters: {'id': 'd4092cc3a5454d64a56e171c8775997b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 13:15:44.499848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:44.503573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 392/802] 10.222.0.15 () {60 vars in 1137 bytes} [Tue Oct 11 13:15:44 2022] POST /identity/v3/domains => generated 190 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:44.506355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.525126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e15332-bf5e-48b7-82bc-83a54133108c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.558883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.559584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.559930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.560232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.560878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:44.561180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.562604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.575161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 411/803] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:44.575981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.588195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.605943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4cac-a293-4fcc-93fd-a6b9709929aa None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.617264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.639987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.640670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.640969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.641238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.651406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.652869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:44.667131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:44.668307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:44.669433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:44.669433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:44.669433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.669433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 393/804] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:44.669433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.681671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7522d-2259-4e6f-9b97-ad33dc87e67b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.715929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.716593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.716863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.717128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.718266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.719694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.721438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 412/805] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:44.722092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.769565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240696c-0b4c-4526-8a8d-042ffef315ba None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.803298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.803933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.804235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.804506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.807076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.811886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.813555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.818345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878941c-9787-4ee6-9f06-8c2c13ad8fe0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:44.824688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:44.825772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:44.826873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:44.827426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:44.827524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 394/806] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:44.827524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.854889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.855437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.855707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.855973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.856646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:44.856937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.858290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.869140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 413/807] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:44.869889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:44.940087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac842c-d6b1-4650-bc8a-7668dcfed698 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:44.977564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:44.978182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:44.978454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:44.978716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:44.979371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:44.980845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:44.985180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8e498-1128-4e00-9f55-59a71e65b3ed None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:45.004470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 395/808] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:15:44 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:45.031420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:45.032193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:45.032532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:45.032865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:45.033661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:45.034062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:45.035745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d8ff-8120-48ef-88e9-dad880610d53 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:45.049824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 414/809] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:45.056440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:45.120502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:45.179957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a8c3-8fa9-4d52-a2e8-87223e395a86 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:45.215954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:45.216656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:45.216931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:45.217205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/db53cf6b98df4bb492c10be037c05ac5` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:45.219531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=edc1cd61b66547b2a388b1ead24a47c1, role_id=db53cf6b98df4bb492c10be037c05ac5)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:45.221697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdad2a0-36c2-45c5-8f4e-19481d5d3ed3 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:45.233974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 396/810] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:45 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/edc1cd61b66547b2a388b1ead24a47c1/roles/db53cf6b98df4bb492c10be037c05ac5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:15:47.173222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062403fc-148e-4373-854a-52c0e22e0c79 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.173787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062403fc-148e-4373-854a-52c0e22e0c79 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.173787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062403fc-148e-4373-854a-52c0e22e0c79 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.175103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 415/811] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:47.180665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732581-2478-4727-8a8a-568d5e84d86d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.181049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732581-2478-4727-8a8a-568d5e84d86d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.181215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732581-2478-4727-8a8a-568d5e84d86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.201163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5f732581-2478-4727-8a8a-568d5e84d86d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:47.202409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5f732581-2478-4727-8a8a-568d5e84d86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:47.280575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 397/812] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:47.319155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f00f-bee1-4e3c-84cc-dc594086926d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.319402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f00f-bee1-4e3c-84cc-dc594086926d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.319671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f00f-bee1-4e3c-84cc-dc594086926d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.321046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 416/813] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:47.326352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.326641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.326969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.345743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:47.346923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:47.382643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:47.415921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 398/814] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:47.547282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f00f-bee1-4e3c-84cc-dc594086926d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:47.576639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:47.655586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f00f-bee1-4e3c-84cc-dc594086926d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:47.676754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:47.689729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:47.690401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.690675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.690937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.701546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:47.706391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:47.707335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:47.708254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 417/815] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:47.713989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:47.779061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be370-5ba5-41a5-a657-40e41042d2e6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:47.797798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:47.812028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:47.812445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c72-e3d7-4ab3-b6f5-10b861a7af38 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:47.812684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.812957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.813224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.823048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:47.827786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:47.828876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:47.829719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 399/816] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:47.834763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:47.848814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:47.849500 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.849779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.850047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.850715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:47.851011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:47.852455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:47.860426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 418/817] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:47.866393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:47.869096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:47.935630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6d85-56a2-41f4-be34-4908c5ae74b2 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:47.962775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d04c41-56c6-468c-a7f4-4c6814260003 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:47.969065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:47.969676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:47.969948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:47.970212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:47.970868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:47.971185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:47.972569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:47.980356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 400/818] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:47.987483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.000177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.000811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.001088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.001377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] PATH_INFO: `/v3/users/a925183b3f63449bb61e3aeecb2ccf1f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.015013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] RBAC: Authorizing `identity:get_user(user_id=a925183b3f63449bb61e3aeecb2ccf1f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.016603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.018614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 419/819] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/users/a925183b3f63449bb61e3aeecb2ccf1f => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:48.024404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.033469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.078592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.085859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bbac1-ef3b-46a3-aa8a-e77904c1759d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.119467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.120093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.120401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.120667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.122209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2d83f-781a-4f91-b6f4-22b64d8d81bd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.132099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.133668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:48.149209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:48.150212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:48.151236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:48.151236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:48.151236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:48.151236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 401/820] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:15:48.156105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.156684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.157324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.157598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.157863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.158941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] RBAC: Authorizing `identity:get_project(project_id=c9452c13896a46f9af3f8dd55b5a73c9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.160396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.162100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 420/821] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:15:48 2022] GET /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:48.168254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.249420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.277249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35013fc3-b3b8-44c6-928a-8c17f63e93d6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.311038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.311704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.312012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.312304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9/users/a925183b3f63449bb61e3aeecb2ccf1f/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.314219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9452c13896a46f9af3f8dd55b5a73c9, user_id=a925183b3f63449bb61e3aeecb2ccf1f, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.315665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.329345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 421/822] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:48 2022] PUT /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9/users/a925183b3f63449bb61e3aeecb2ccf1f/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:15:48.373159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90658c-17ed-4e53-b23e-f557e7eff076 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.410189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.410805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.411075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.411342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.412051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:48.412384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.413773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.426834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 402/823] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:15:48.432867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.538946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cb081-9106-45d1-83e6-7a6c0e9fd05f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.574043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.574695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.574972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.575235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.577431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.584910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.586835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:15:48.599603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:48.600762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:48.601881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 422/824] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 11 13:15:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:48.606712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.615746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.707196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80808056-5a64-439d-94e7-4bb111df8cff None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.741334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.741941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.742166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.742468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.743189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:48.743489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.744976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:48.762254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 403/825] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 11 13:15:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:48.768153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:48.874091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8551603-408c-450c-bb74-5da8d760b3a8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:48.908534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:48.909140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:48.909409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:48.909675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:48.969796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:48.978397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:48.979851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.078470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 423/826] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:49.084584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:49.146499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:49.187623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21cd78-a939-4fc2-b954-295ff4c7f282 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:49.222705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:49.223325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:49.223594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:49.223855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:49.234167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:49.235588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:15:49.250109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:49.251082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:15:49.251995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:15:49.251995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:15:49.251995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:49.251995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 404/827] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:49.257207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:49.349414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:49.356545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ac4a-3aed-4f98-8f7b-65caf67c1dbb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:49.390487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:49.391092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:49.391361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:49.391626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:49.392288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:49.392584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:49.393943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.405231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 424/828] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:49.410972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:49.510399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fb361-16bb-44f0-b557-d4aceac207fb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:49.527882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:49.545429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:49.546052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:49.546323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:49.546582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:49.553254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:49.554668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:15:49.565607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:49.566533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:15:49.567417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 405/829] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:49.572864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:49.676625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35181e-fa7e-48bd-a7a1-bccb5be45bf2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:49.711142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:49.711733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:49.711995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:49.712273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:49.712986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:49.713285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:49.714995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329370d-b148-40d6-8f21-b890d8d29ed8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.727485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 425/830] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:49.733851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:49.833335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7741756-43b1-4075-a215-b1ca91744c2c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:49.867655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:49.868285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:49.868554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:49.868821 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:49.869538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] RBAC: Adding query filter params (role.id=db53cf6b98df4bb492c10be037c05ac5, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=edc1cd61b66547b2a388b1ead24a47c1) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:49.869839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:49.871243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1060f37-760b-4263-b995-309af69aeefa None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:49.877094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 406/831] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:49 2022] GET /identity/v3/role_assignments?user.id=edc1cd61b66547b2a388b1ead24a47c1&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=db53cf6b98df4bb492c10be037c05ac5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:50.444661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf701-048b-4e0f-a61e-88997f686323 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:50.445227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf701-048b-4e0f-a61e-88997f686323 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:50.445227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf701-048b-4e0f-a61e-88997f686323 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:50.446544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 426/832] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:50.452038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:50.452509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:50.452611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:50.471173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:50.472400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:50.507587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:50.542009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 407/833] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:50.802054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcf701-048b-4e0f-a61e-88997f686323 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:50.819664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:50.920748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcf701-048b-4e0f-a61e-88997f686323 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:50.960032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:50.960726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:50.961003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:50.961273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:50.972493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:15:50.977343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:15:50.978445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:15:50.979427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 427/834] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:15:50.985299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:51.092085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f69283-51bf-4e0c-b80c-6879f9de461b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:51.126798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:51.127389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:51.127658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:51.127922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:51.128634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:51.128936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:51.130278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:51.138789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 408/835] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:51.148541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:51.240010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:51.269640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f3308-da85-47af-8157-4c158d2a3233 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:51.303791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:51.304480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:51.304707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:51.304967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] PATH_INFO: `/v3/users/a925183b3f63449bb61e3aeecb2ccf1f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:51.306093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] RBAC: Authorizing `identity:get_user(user_id=a925183b3f63449bb61e3aeecb2ccf1f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:51.307508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:51.309079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 428/836] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:15:51 2022] GET /identity/v3/users/a925183b3f63449bb61e3aeecb2ccf1f => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:51.315081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:51.348842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:51.418828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13825655-3301-49fd-b8a0-0bdf2f772370 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:51.452854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:51.453462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:51.453703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:51.453972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:51.520348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:51.522138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:51.553452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:51.625599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 409/837] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:15:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:15:51.631669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:51.739495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ed290-a6ed-46b3-b499-47f832cc9fdf None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:51.774521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:51.775139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:51.775414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:51.775673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:51.776820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] RBAC: Authorizing `identity:get_project(project_id=c9452c13896a46f9af3f8dd55b5a73c9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:51.778271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad4f2c-2da1-42bd-b6ba-3096e0d70680 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:51.779902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 429/838] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:15:51 2022] GET /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:51.786365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:51.883698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcec5b-3b01-434b-ae40-48017eaeb01f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:51.913320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:51.919557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:51.920151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:51.920460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:51.920725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:51.921452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=c9452c13896a46f9af3f8dd55b5a73c9, user.id=a925183b3f63449bb61e3aeecb2ccf1f) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:15:51.921755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:51.923164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf990367-f05c-4c95-a744-ad515fd17a02 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:51.929860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 410/839] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:15:51 2022] GET /identity/v3/role_assignments?user.id=a925183b3f63449bb61e3aeecb2ccf1f&scope.project.id=c9452c13896a46f9af3f8dd55b5a73c9&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:54.020774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e089a-c8e5-4b6e-b01f-a57f2b09beda None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:54.021620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e089a-c8e5-4b6e-b01f-a57f2b09beda None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:54.021620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e089a-c8e5-4b6e-b01f-a57f2b09beda None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:54.023996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 430/840] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:54.032934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:54.033339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:54.033539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:54.053837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:54.055039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:54.129703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 411/841] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:54.395268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e089a-c8e5-4b6e-b01f-a57f2b09beda None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:54.416332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:54.511155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e089a-c8e5-4b6e-b01f-a57f2b09beda None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:54.546322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:54.546948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:54.547222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:54.547485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:54.548728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:54.550170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a0f7-0944-4ac1-8c65-afc21d7b1f9a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:54.551730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 431/842] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:15:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:54.561066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:54.668032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf91af-46e8-4e01-bed7-71dd4b39b65f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:54.693871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:54.710142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:54.711037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:54.711409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:54.711780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:54.712815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:54.715113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f798-2d29-4da2-a949-cd44a426dd3d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:54.740801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 412/843] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:15:54 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:15:56.866243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d4ce-64f1-423a-9d1c-44f8102b13c7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:56.866772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d4ce-64f1-423a-9d1c-44f8102b13c7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:56.866772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d4ce-64f1-423a-9d1c-44f8102b13c7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:56.868046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 432/844] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:15:56.874321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:56.875016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:56.875213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:56.896517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:15:56.898029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:15:56.967663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 413/845] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:15:57.234382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9d4ce-64f1-423a-9d1c-44f8102b13c7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:57.332737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:57.368942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9d4ce-64f1-423a-9d1c-44f8102b13c7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:57.406478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:57.407110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:57.407385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:57.407653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] PATH_INFO: `/v3/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:57.409079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] RBAC: Authorizing `identity:get_role(role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:57.410463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:57.411793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 433/846] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:15:57 2022] GET /identity/v3/roles/314f2033c1db4762926d37c1532eb2f1 => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:57.417911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:57.531321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f11fd-7f16-4afa-9a5d-107b1eaa04b9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:57.544506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:15:57.566146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:57.566779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:57.567054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:57.567324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] PATH_INFO: `/v3/users/f224a1484ffb439ab8b18f000792b6fe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:57.582086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] RBAC: Authorizing `identity:get_user(user_id=f224a1484ffb439ab8b18f000792b6fe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:57.583562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:57.585218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 414/847] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:15:57 2022] GET /identity/v3/users/f224a1484ffb439ab8b18f000792b6fe => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:57.591735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:57.712759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50915bb9-e52a-4589-8e38-ff2b4fa49714 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:57.749432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:57.750048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:57.750321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:57.750589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:57.751823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] RBAC: Authorizing `identity:get_project(project_id=c9452c13896a46f9af3f8dd55b5a73c9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:57.753314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b0b22-75c0-4f8a-9b4b-03a3eb41ba90 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:57.754949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 434/848] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:15:57 2022] GET /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:15:57.764312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:15:57.882442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1692b1f-e6e7-4ef2-a97a-6764f07d643b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:15:57.917881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:15:57.918525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:15:57.918802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:15:57.919067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9/users/f224a1484ffb439ab8b18f000792b6fe/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:15:57.921391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] RBAC: Authorizing `identity:create_grant(project_id=c9452c13896a46f9af3f8dd55b5a73c9, user_id=f224a1484ffb439ab8b18f000792b6fe, role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:15:57.922800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247625-3e14-4bcf-a334-e80729e3e78c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:15:57.941444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 415/849] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:15:57 2022] PUT /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9/users/f224a1484ffb439ab8b18f000792b6fe/roles/314f2033c1db4762926d37c1532eb2f1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:16:00.044449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77299-7ede-4ab1-9009-1dbb70e004df None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:00.045007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77299-7ede-4ab1-9009-1dbb70e004df None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:00.045007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77299-7ede-4ab1-9009-1dbb70e004df None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:00.046359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 435/850] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:00.052857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:00.053557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:00.053724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:00.077682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:00.078879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:00.102765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:00.148234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 416/851] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:00.419046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77299-7ede-4ab1-9009-1dbb70e004df None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:00.437125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:00.528238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77299-7ede-4ab1-9009-1dbb70e004df None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:00.564313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:00.564938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:00.565213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:00.565483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] PATH_INFO: `/v3/roles/314f2033c1db4762926d37c1532eb2f1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:00.566965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] RBAC: Authorizing `identity:get_role(role_id=314f2033c1db4762926d37c1532eb2f1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:00.568433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:00.569764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 436/852] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:00 2022] GET /identity/v3/roles/314f2033c1db4762926d37c1532eb2f1 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:00.575881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:00.676168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d7fad-1656-4f75-b49d-088260008e78 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:00.711516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:00.712163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:00.712469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:00.712741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] PATH_INFO: `/v3/users/f224a1484ffb439ab8b18f000792b6fe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:00.713987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] RBAC: Authorizing `identity:get_user(user_id=f224a1484ffb439ab8b18f000792b6fe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:00.715427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:00.717049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 417/853] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:00 2022] GET /identity/v3/users/f224a1484ffb439ab8b18f000792b6fe => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:00.725765 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:00.813286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:00.832362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2c3a-a21d-4717-b53c-b30058f0e59b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:00.989710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:00.990202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:00.990473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:00.990741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:01.062601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:01.064025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:01.146482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:01.166874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 437/854] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:16:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:16:01.175907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:01.204932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:01.281277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498eba9-3a8f-43d9-aea7-f5697fdc7970 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:01.318222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:01.318847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:01.319125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:01.319388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] PATH_INFO: `/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:01.320566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] RBAC: Authorizing `identity:get_project(project_id=c9452c13896a46f9af3f8dd55b5a73c9)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:01.321980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81c2c-27f5-4287-9684-a8e6c14084a2 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:01.323580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 418/855] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:16:01 2022] GET /identity/v3/projects/c9452c13896a46f9af3f8dd55b5a73c9 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:01.332951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:01.434348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ce04-162d-4ae7-b9ca-3a2381ce0f8f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:01.469231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:01.469826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:01.470098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:01.470366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:01.471089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] RBAC: Adding query filter params (role.id=314f2033c1db4762926d37c1532eb2f1, scope.project.id=c9452c13896a46f9af3f8dd55b5a73c9, user.id=f224a1484ffb439ab8b18f000792b6fe) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:01.471400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:01.472847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc8adc-c41f-4125-b2e6-2be9a886d2e0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:01.481513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 438/856] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:16:01 2022] GET /identity/v3/role_assignments?user.id=f224a1484ffb439ab8b18f000792b6fe&scope.project.id=c9452c13896a46f9af3f8dd55b5a73c9&role.id=314f2033c1db4762926d37c1532eb2f1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:16:03.600971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcd8bd-58ee-48e8-a5ee-14e2299559e8 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:03.601415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcd8bd-58ee-48e8-a5ee-14e2299559e8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:03.601524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcd8bd-58ee-48e8-a5ee-14e2299559e8 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:03.602882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 419/857] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:03.608393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:03.608820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:03.608909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:03.627589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:03.628828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:03.684400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:03.709556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 439/858] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:03.973644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcd8bd-58ee-48e8-a5ee-14e2299559e8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:04.074641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:04.083405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcd8bd-58ee-48e8-a5ee-14e2299559e8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:04.118979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:04.119597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:04.119871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:04.120160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:04.121315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:04.122731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b5d0-2118-44cd-bc33-f9e772f4fdde None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:04.124294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 420/859] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:04.130759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:04.244761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b10b3-3086-42ea-9363-42fb4da1b768 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:04.283925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:04.284560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:04.284831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:04.285096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:04.285737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:04.287188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba668b-cd69-4877-9af9-2544dddf05aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:04.314688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 440/860] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:16:04 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:16:06.488905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2a61-27f8-404e-b831-a23200fd5e5e None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:06.489952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2a61-27f8-404e-b831-a23200fd5e5e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:06.489952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2a61-27f8-404e-b831-a23200fd5e5e None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:06.492048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 421/861] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:06.498426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:06.498867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:06.499009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:06.526310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:06.527485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:06.550477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:06.598276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 441/862] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:06.861595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca2a61-27f8-404e-b831-a23200fd5e5e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:06.970708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca2a61-27f8-404e-b831-a23200fd5e5e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:07.005560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:07.006180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:07.006464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:07.006734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:07.007975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:07.009445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac045b-008d-4c59-9bcf-8ca1de070303 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:07.011001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 422/863] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:07.017081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:07.135736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d37e7-9c3e-41b5-800b-2a871606544b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:07.170587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:07.171193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:07.171477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:07.171735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:07.172410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:07.173862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125dbf0-c1bf-43b5-982d-f098fe6b01f7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:07.197866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 442/864] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:16:07 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:16:09.330402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde199-1984-4909-bd3b-4d418eaa5762 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:09.331310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde199-1984-4909-bd3b-4d418eaa5762 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:09.331310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde199-1984-4909-bd3b-4d418eaa5762 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:09.333368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 423/865] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:09.341995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:09.342403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:09.342593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:09.369006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:09.370220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:09.379625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:09.445907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 443/866] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:09.713372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde199-1984-4909-bd3b-4d418eaa5762 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:09.754255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:09.850544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde199-1984-4909-bd3b-4d418eaa5762 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:09.889229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:09.889856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:09.890125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:09.890390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:09.900799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:16:09.905567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:16:09.906688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:16:09.907776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 424/867] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:16:09.913866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:10.014473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4447b-2d18-47a3-aa1d-8f9baa46c190 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:10.049697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:10.050294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:10.050562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:10.050824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:10.051478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:10.051831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:10.053309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:10.061217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 444/868] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:10.067592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:10.169047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:10.180911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c198cf1-24f7-4e3c-a14f-4abb18eb4d4f None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:10.215790 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:10.216437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:10.216714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:10.216985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] PATH_INFO: `/v3/users/2f4fafc7dcce4aa58e6ccecc238e13fa` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:10.230270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] RBAC: Authorizing `identity:get_user(user_id=2f4fafc7dcce4aa58e6ccecc238e13fa)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:10.231695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:10.233301 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 425/869] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:10 2022] GET /identity/v3/users/2f4fafc7dcce4aa58e6ccecc238e13fa => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:10.242038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:10.256405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:10.351517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb06fd-40ac-4d21-b54d-cbb162e80273 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:10.385723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:10.386337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:10.386619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:10.386894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] PATH_INFO: `/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:10.388156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] RBAC: Authorizing `identity:get_project(project_id=e4affbbc0f4345b49fb6c0ba4c4d58fb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:10.389644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42fb27-7022-4784-9c39-6e89c3754e16 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:10.391262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 445/870] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:16:10 2022] GET /identity/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:10.398464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:10.508304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6638c51-5103-4c33-9137-f3409d14fa01 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:10.544009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:10.544683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:10.544958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:10.545229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] PATH_INFO: `/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb/users/2f4fafc7dcce4aa58e6ccecc238e13fa/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:10.547397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4affbbc0f4345b49fb6c0ba4c4d58fb, user_id=2f4fafc7dcce4aa58e6ccecc238e13fa, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:10.548839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe448a-87f4-4359-aa62-50965ac8b957 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:10.564465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 426/871] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:16:10 2022] PUT /identity/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb/users/2f4fafc7dcce4aa58e6ccecc238e13fa/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:16:12.652933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce48c-af3d-4692-815a-eda5487047e5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:12.653432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce48c-af3d-4692-815a-eda5487047e5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:12.653432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce48c-af3d-4692-815a-eda5487047e5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:12.654817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 446/872] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:12.661373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:12.661809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:12.661898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:12.681471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:12.682682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:12.685739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:12.756152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 427/873] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:13.018255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ce48c-af3d-4692-815a-eda5487047e5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:13.098811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:13.125818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ce48c-af3d-4692-815a-eda5487047e5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:13.162216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:13.162833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:13.163105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:13.163369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:13.174607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:16:13.179412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:16:13.180545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:16:13.181548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 447/874] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:16:13.187309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:13.293238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf261c-9cfa-45a7-bf78-6fd7575498af None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:13.330450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:13.331047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:13.331320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:13.331588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:13.332289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:13.332589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:13.333947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:13.342612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 428/875] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:13.351453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:13.470518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd565-37a0-4d04-9c3b-26116cad2a53 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:13.505437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:13.506053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:13.506323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:13.506590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] PATH_INFO: `/v3/users/2f4fafc7dcce4aa58e6ccecc238e13fa` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:13.507819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] RBAC: Authorizing `identity:get_user(user_id=2f4fafc7dcce4aa58e6ccecc238e13fa)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:13.509283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:13.510832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 448/876] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/users/2f4fafc7dcce4aa58e6ccecc238e13fa => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:13.516617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:13.519549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:13.624588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8375-f0c2-4c96-8cb5-635ba6dc8788 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:13.658969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:13.659560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:13.659830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:13.660096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:13.726626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:13.728044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:13.729865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:13.836401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 429/877] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 13:16:13.842747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:13.845694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:13.946687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a484db-5675-4745-ac38-e02829a09615 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:13.983874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:13.984510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:13.984781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:13.985046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] PATH_INFO: `/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:13.986254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] RBAC: Authorizing `identity:get_project(project_id=e4affbbc0f4345b49fb6c0ba4c4d58fb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:13.987668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0965b9c-1da0-4cee-a3bf-66e9448d3752 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:13.989382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 449/878] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/projects/e4affbbc0f4345b49fb6c0ba4c4d58fb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:13.995546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:14.097171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a814c3-4ce4-43b5-b011-9ca9e37a8817 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:14.098673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:14.133601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:14.134187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:14.134464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:14.134730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:14.135453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=e4affbbc0f4345b49fb6c0ba4c4d58fb, user.id=2f4fafc7dcce4aa58e6ccecc238e13fa) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:14.135757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:14.137197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81fcee-c613-432c-83a2-b763921dfcd6 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:14.142975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 430/879] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:16:13 2022] GET /identity/v3/role_assignments?user.id=2f4fafc7dcce4aa58e6ccecc238e13fa&scope.project.id=e4affbbc0f4345b49fb6c0ba4c4d58fb&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:16.271470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ffd12-3b90-4a63-98b6-408bef448d3f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:16.272398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ffd12-3b90-4a63-98b6-408bef448d3f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:16.272398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ffd12-3b90-4a63-98b6-408bef448d3f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:16.273712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 450/880] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:16.280154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd755d-0be4-4077-bea8-b6325c7cf1e9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:16.280864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd755d-0be4-4077-bea8-b6325c7cf1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:16.280997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd755d-0be4-4077-bea8-b6325c7cf1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:16.312914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-38bd755d-0be4-4077-bea8-b6325c7cf1e9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:16.314719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-38bd755d-0be4-4077-bea8-b6325c7cf1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:16.393693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 431/881] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:16.662179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ffd12-3b90-4a63-98b6-408bef448d3f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:16.739161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:16.772450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ffd12-3b90-4a63-98b6-408bef448d3f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:16.807185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094c464-f79a-491a-92d6-168533121492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:16.807785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094c464-f79a-491a-92d6-168533121492 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:16.808059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094c464-f79a-491a-92d6-168533121492 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:16.808357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094c464-f79a-491a-92d6-168533121492 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:16.809029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094c464-f79a-491a-92d6-168533121492 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:16.810393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094c464-f79a-491a-92d6-168533121492 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:16.834112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 451/882] 10.222.0.15 () {60 vars in 1138 bytes} [Tue Oct 11 13:16:16 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:16:18.982549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b1de-09b8-4e15-a201-873bcd7c21ab None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:18.983390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b1de-09b8-4e15-a201-873bcd7c21ab None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:18.983390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59b1de-09b8-4e15-a201-873bcd7c21ab None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:18.985555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 432/883] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:18.993269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:18.993708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:18.993810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:19.022092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:19.023269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:19.095854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 452/884] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:19.360244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59b1de-09b8-4e15-a201-873bcd7c21ab None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:19.459616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:19.468580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59b1de-09b8-4e15-a201-873bcd7c21ab None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:19.504277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:19.504964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:19.505240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:19.505511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] PATH_INFO: `/v3/domains/d61e635bc11a4237a4cd811119a424a4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:19.516329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d61e635bc11a4237a4cd811119a424a4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:19.517763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7708b8-25f3-44e9-8f59-f56a9f0f8787 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:19.519308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 433/885] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 11 13:16:19 2022] GET /identity/v3/domains/d61e635bc11a4237a4cd811119a424a4 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:19.528392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:19.588728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:19.630169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee189b-b152-44b0-96c9-f064cefaa4f2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:19.664497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:19.665094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:19.665363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:19.665625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:19.666274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:19.667682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe57c0-8f6f-418b-a19a-d6f2362dabf0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:19.699045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 453/886] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:16:19 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:16:21.851630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835daf3d-334a-4b4a-9c01-e9a7c7afb826 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:21.852514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835daf3d-334a-4b4a-9c01-e9a7c7afb826 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:21.852514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835daf3d-334a-4b4a-9c01-e9a7c7afb826 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:21.854609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 434/887] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:21.859989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274612-5795-487a-beed-44e02a453275 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:21.860448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274612-5795-487a-beed-44e02a453275 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:21.860547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b274612-5795-487a-beed-44e02a453275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:21.882019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1b274612-5795-487a-beed-44e02a453275 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:21.883218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b274612-5795-487a-beed-44e02a453275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:21.954262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 454/888] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:22.218048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835daf3d-334a-4b4a-9c01-e9a7c7afb826 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:22.330664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835daf3d-334a-4b4a-9c01-e9a7c7afb826 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:22.373804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:22.374807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:22.375261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:22.375702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] PATH_INFO: `/v3/domains/d61e635bc11a4237a4cd811119a424a4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:22.377803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] RBAC: Authorizing `identity:get_domain(domain_id=d61e635bc11a4237a4cd811119a424a4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:22.380060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d661df7-524c-4e01-98b7-76b83440077b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:22.381667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 435/889] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 11 13:16:22 2022] GET /identity/v3/domains/d61e635bc11a4237a4cd811119a424a4 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:22.387745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b274612-5795-487a-beed-44e02a453275 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:22.434410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:22.493405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b274612-5795-487a-beed-44e02a453275 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:22.527654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:22.528273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:22.528544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:22.528809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:22.529440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:22.530891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefe8b1-958f-49e8-9c4d-26fc0b3b0690 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:22.553768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 455/890] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:16:22 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:16:24.670354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484781c-d847-470c-97e7-55c21f1524fe None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:24.671187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484781c-d847-470c-97e7-55c21f1524fe None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:24.671187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484781c-d847-470c-97e7-55c21f1524fe None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:24.673371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 436/891] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:24.682396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:24.682822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:24.682960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:24.704879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:24.706063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:24.775346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 456/892] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:25.039971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484781c-d847-470c-97e7-55c21f1524fe None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:25.069512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:25.148402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484781c-d847-470c-97e7-55c21f1524fe None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:25.182599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:25.183211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:25.183478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:25.183742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:25.194388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:16:25.199133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:16:25.200315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:25.201373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:16:25.204294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 437/893] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:16:25.206619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:25.240518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:25.321887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933e1a-47ff-4550-bfb6-dd486d24bf02 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:25.357019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:25.357621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:25.357898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:25.358161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:25.358827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:25.359130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:25.360526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:25.368863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 457/894] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:25.375008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:25.471737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:25.489322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67712-e1c6-4978-be44-1cc49a754dce None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:25.533990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:25.534886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:25.535278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:25.535606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] PATH_INFO: `/v3/users/1a80e473f6ec4d8e8aee767cf17a9407` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:25.549162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] RBAC: Authorizing `identity:get_user(user_id=1a80e473f6ec4d8e8aee767cf17a9407)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:25.550577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:25.552209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 438/895] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:25 2022] GET /identity/v3/users/1a80e473f6ec4d8e8aee767cf17a9407 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:16:25.558106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:25.652203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:25.662052 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6263ac2-1487-40da-bd98-4df61c03f2e9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:25.696730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:25.697325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:25.697610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:25.697873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] PATH_INFO: `/v3/projects/1fcb5336741a4a869b7c9f7bddce3271` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:25.698995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcb5336741a4a869b7c9f7bddce3271)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:25.700449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa92bf-f799-484b-803c-d0bc87bf4c24 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:25.702060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 458/896] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:16:25 2022] GET /identity/v3/projects/1fcb5336741a4a869b7c9f7bddce3271 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:25.710440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:25.818635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a444c-29f0-4a2a-99bd-465d6f996088 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:25.858094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:25.858664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:25.858935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:25.859201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] PATH_INFO: `/v3/projects/1fcb5336741a4a869b7c9f7bddce3271/users/1a80e473f6ec4d8e8aee767cf17a9407/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:25.861379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcb5336741a4a869b7c9f7bddce3271, user_id=1a80e473f6ec4d8e8aee767cf17a9407, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:25.862778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727e941-5958-4fc3-a56f-08bda3f02778 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:25.885482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 439/897] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:16:25 2022] PUT /identity/v3/projects/1fcb5336741a4a869b7c9f7bddce3271/users/1a80e473f6ec4d8e8aee767cf17a9407/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:16:27.986842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8208da-db56-4de5-a24e-482239b454fa None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:27.987687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8208da-db56-4de5-a24e-482239b454fa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:27.987687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8208da-db56-4de5-a24e-482239b454fa None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:27.989545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 459/898] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:16:27.995001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:27.995420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:27.995537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:28.002415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:28.016715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:16:28.017901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:16:28.087178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 440/899] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:16:28.349860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8208da-db56-4de5-a24e-482239b454fa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:28.469432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8208da-db56-4de5-a24e-482239b454fa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:28.514693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:28.515314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:28.515593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:28.515860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:28.526903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:16:28.531674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:16:28.532724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:16:28.533775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 460/900] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:16:28.539763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:28.657993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d716-c73b-499e-b04f-fa7763d035d4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:28.688468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:28.694586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:28.695182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:28.695450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:28.695713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:28.696380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:28.696677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:28.698038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:28.706103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 441/901] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:28.712860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:28.732801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:28.817911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751add6a-3397-453d-bfe7-171ed5181b7e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:28.851870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:28.852510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:28.852786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:28.853055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] PATH_INFO: `/v3/users/1a80e473f6ec4d8e8aee767cf17a9407` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:28.854266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] RBAC: Authorizing `identity:get_user(user_id=1a80e473f6ec4d8e8aee767cf17a9407)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:28.855694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:28.857269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 461/902] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 11 13:16:28 2022] GET /identity/v3/users/1a80e473f6ec4d8e8aee767cf17a9407 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:28.865773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:28.970382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c985f-97d2-45ef-965d-1a2d3c8eb42a None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:29.006279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:29.006899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:29.007140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:29.007403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:29.069013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:29.077127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:29.078555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:29.186598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 442/903] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:16:29.195001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:29.285974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:29.301963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbfe5a-9bf1-4870-89a8-46112e176644 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:29.337753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:29.338374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:29.338653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:29.338925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] PATH_INFO: `/v3/projects/1fcb5336741a4a869b7c9f7bddce3271` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:29.340152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] RBAC: Authorizing `identity:get_project(project_id=1fcb5336741a4a869b7c9f7bddce3271)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:29.341599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735901-93de-495c-9722-da1cbdea60ad None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:29.343217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 462/904] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 11 13:16:29 2022] GET /identity/v3/projects/1fcb5336741a4a869b7c9f7bddce3271 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:29.349508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:16:29.440786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:29.448304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220b41-614e-4275-ba81-bea92f390ff3 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:16:29.483179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:16:29.483765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:16:29.484036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:16:29.484338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:16:29.485057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=1fcb5336741a4a869b7c9f7bddce3271, user.id=1a80e473f6ec4d8e8aee767cf17a9407) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:16:29.485360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:16:29.486764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af492b-fc26-4a8d-8661-02b8e0839436 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:16:29.493052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 443/905] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:16:29 2022] GET /identity/v3/role_assignments?user.id=1a80e473f6ec4d8e8aee767cf17a9407&scope.project.id=1fcb5336741a4a869b7c9f7bddce3271&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:16:39.332596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=23 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:16:39.489971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=21 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:24.175066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536784-32cf-4efb-9ad9-5dbcd7e6c5dd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:24.175901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536784-32cf-4efb-9ad9-5dbcd7e6c5dd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:24.175901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536784-32cf-4efb-9ad9-5dbcd7e6c5dd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:24.177640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 463/906] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:24.183386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:24.183954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:24.183954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:24.203778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:24.205023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:24.287827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 444/907] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:24.553013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48536784-32cf-4efb-9ad9-5dbcd7e6c5dd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:24.676419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48536784-32cf-4efb-9ad9-5dbcd7e6c5dd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:24.716786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:24.717457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:24.717809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:24.718146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:24.738634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:24.740414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:24.759065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:24.768380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:24.769438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:24.769438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:24.769438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:24.769438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 464/908] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:24.769684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:24.900015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691da55-cec2-45fe-82b3-3815da2bc3e3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:24.934252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:24.944069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:24.945159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:24.945585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:24.945986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:24.946845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:24.947304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:24.948922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d34f39-4995-4063-8f81-5bb96fff8025 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:24.962314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 445/909] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:24.970892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:25.040343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:25.103027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cee45f-d127-47b2-8e48-65080845c902 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:25.150230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:25.150935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:25.151326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:25.151697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:25.152540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:25.159154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd27cf-568a-4019-a277-7a7fa8cc8f99 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:25.188164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 465/910] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 11 13:17:24 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:17:27.368997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832dd5c-e4c7-48c8-8c4a-863849b898e6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:27.369568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832dd5c-e4c7-48c8-8c4a-863849b898e6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:27.369568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832dd5c-e4c7-48c8-8c4a-863849b898e6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:27.370887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 446/911] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:27.379594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:27.379594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:27.379594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:27.396606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:27.397800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:27.472263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 466/912] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:27.738421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832dd5c-e4c7-48c8-8c4a-863849b898e6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:27.863038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832dd5c-e4c7-48c8-8c4a-863849b898e6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:27.903942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:27.904742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:27.905024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:27.905369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:27.921282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:17:27.927363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:27.928666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:27.929536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 447/913] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:17:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:27.934743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:27.981087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:28.056575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f8209-07d1-46df-b558-1b8877cb5191 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:28.097322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:28.098041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:28.098390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:28.098678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:28.099465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:28.099815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:28.101715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:28.116420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 467/914] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:17:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:28.118247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:28.185597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:28.250064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dc26-ccb4-4b28-9ad1-ba8e46a755fb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:28.295164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:28.296168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:28.296573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:28.297070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:28.312081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:28.313958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:28.331666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:28.333742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:28.334642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:28.334642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:28.334642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:28.334642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 448/915] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:28.341144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:28.469964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:28.471836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eed1a0-5649-4869-9ae1-87521fa653ae None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:28.514666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:28.515430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:28.515774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:28.516132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:28.516976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:28.517490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:28.519125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:28.538049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 468/916] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:28.539497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:28.664062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb77266-db19-4900-ae70-a06955ae33b1 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:28.666093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:28.710799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:28.711635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:28.712013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:28.712434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:28.721267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:28.722938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:17:28.736789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:28.738126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:28.739439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:28.744914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 449/917] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:17:28.745135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:28.868068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93bdb3-6614-4a95-985a-57133580a592 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:28.906766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:28.907709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:28.908238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:28.908703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:28.909668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:28.910191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:28.911913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:28.934758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 469/918] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 11 13:17:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:28.941043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:29.061263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc93-fa84-4a09-999a-a0a6a3fd3c27 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:29.110422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:29.111186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:29.111611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:29.112106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:29.125259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:29.126774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:29.141713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:29.142654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:29.144330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:29.144330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:29.144330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:29.144330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 450/919] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:29.149174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:29.170009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:29.261585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e017-6b46-49a8-ba24-6ab8a30d7ce7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:29.307293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:29.308043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:29.308379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:29.308673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:29.309443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:29.309770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:29.311420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:29.324312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 470/920] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:29.329878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:29.348578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:29.441863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba05748-c5d0-4c7a-af3b-692f733388d8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:29.477544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:29.478523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:29.478870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:29.479275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:29.486901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:29.488459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:17:29.501336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:29.502646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:29.503787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:29.509168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 451/921] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:29.510489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:29.603761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:29.626688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f695d-7713-48c8-a9b8-46596b3bd6d2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:29.665506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:29.666341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:29.666741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:29.667143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:29.667964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:29.668426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:29.669883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d9efc-0b43-4da2-8408-4984e578d1cb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:29.683199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 471/922] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:29.690536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:29.776430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:29.805863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97886d-2211-4c4f-b4d9-69625fc895ff None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:29.840427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:29.841107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:29.841379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:29.841650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/eabb880b598b47c0b7d7f969d4eebffb/roles/8797ad8b8aa84c2c9c388b1deea4d285` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:29.856481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=eabb880b598b47c0b7d7f969d4eebffb, role_id=8797ad8b8aa84c2c9c388b1deea4d285)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:29.857909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ed4d3-ac5d-4d64-8ef3-5ee266a5c33b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:29.870322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 452/923] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:17:29 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/eabb880b598b47c0b7d7f969d4eebffb/roles/8797ad8b8aa84c2c9c388b1deea4d285 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:17:32.093126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3598-bc48-4263-b7ad-071ac7df7289 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:32.093688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3598-bc48-4263-b7ad-071ac7df7289 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:32.093688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3598-bc48-4263-b7ad-071ac7df7289 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:32.095007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 472/924] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:32.100672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:32.101161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:32.101251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:32.119642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:32.120889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:32.188192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 453/925] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 13:17:32.454082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc3598-bc48-4263-b7ad-071ac7df7289 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:32.554860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc3598-bc48-4263-b7ad-071ac7df7289 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:32.589900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:32.590598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:32.590828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:32.591085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:32.598053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:32.602749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:17:32.609096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:32.610044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:32.610880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 473/926] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 11 13:17:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:32.616708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:32.674879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:32.718786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7e276-c5d4-4968-9c53-c38dca790887 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:32.754164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:32.754777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:32.755060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:32.755340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:32.755980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] RBAC: Adding query filter params (name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:32.756309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:32.757709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:32.765685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 454/927] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 11 13:17:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:32.773113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:32.887524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee958-980d-477a-ab33-4f7ad2c5e8d7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:32.924798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:32.925521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:32.925866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:32.926209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:32.938888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:32.940554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:32.956800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:32.957905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:32.964600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:32.964600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:32.964600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:32.964600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 474/928] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:32.964875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:33.073104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6c541-d81b-4133-a9de-8c6cca94008c None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:33.091804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:33.112200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:33.112874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:33.113159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:33.113463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:33.114158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:33.114502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:33.115973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:33.129958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 455/929] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:33.135934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:33.191644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:33.248868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb981a68-2eed-48d2-8959-2af8eb444597 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:33.289591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:33.290288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:33.290665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:33.291011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:33.301577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:33.303352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:17:33.319238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:33.320277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:33.321238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 475/930] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:33.327735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:33.427948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d48e15-b99c-4369-9b35-e3d7965fed57 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:33.464363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:33.464990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:33.465282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:33.465539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:33.466316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:33.466547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:33.467980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:33.485360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 456/931] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 11 13:17:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:33.492201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:33.603293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d83b72-8d2f-4f00-ba28-d2819d081efa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:33.613900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:33.639255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:33.639869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:33.640171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:33.640443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:33.715739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:33.717246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:33.822499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 476/932] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:17:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:17:33.828842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:33.864367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:33.936883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a51d8-25fe-49bf-94b9-eb2a5ea13cd8 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:33.972748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:33.973391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:33.973663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:33.973931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:33.984306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:33.985745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:34.000271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:34.001746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:34.003033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:34.003033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:34.003033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:34.003033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 457/933] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:17:34.007478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:34.019680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:34.113979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b9e7-b72e-4203-88e4-9c43a9eee636 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:34.152349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:34.153158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:34.153253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:34.153450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:34.154131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:34.154422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:34.155969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:34.169016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 477/934] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:34.175234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:34.265859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:34.284544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947507c-e506-4af9-8387-f1e3d6b00a5e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:34.323757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:34.324615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:34.324954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:34.325254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:34.333003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:34.334625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:17:34.348309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:34.349393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:34.350316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 458/935] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:34.355727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:34.432928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:34.468858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81916600-284d-4959-9f09-dd470fcccd07 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:34.504069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:34.505106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:34.505420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:34.505650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:34.506344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:34.506645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:34.508081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abc25b-6797-46d4-b60e-ab3663bbdc2a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:34.520918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 478/936] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:34.527335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:34.645999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99067ceb-1f6f-434b-91f1-95c9f443d261 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:34.680770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:34.681446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:34.681719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:34.681986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:34.682837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] RBAC: Adding query filter params (role.id=8797ad8b8aa84c2c9c388b1deea4d285, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=eabb880b598b47c0b7d7f969d4eebffb) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:34.683224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:34.684655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b074d9-7e20-4e08-9b87-d1824936b634 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:34.690419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 459/937] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:17:34 2022] GET /identity/v3/role_assignments?user.id=eabb880b598b47c0b7d7f969d4eebffb&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=8797ad8b8aa84c2c9c388b1deea4d285 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:36.849243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f305-5c32-4904-9a94-7acf1a523dfc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:36.849798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f305-5c32-4904-9a94-7acf1a523dfc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:36.849798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f305-5c32-4904-9a94-7acf1a523dfc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:36.851085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 479/938] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:36.860148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:36.860148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:36.860148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:36.877502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:36.878680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:36.881520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:36.948472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 460/939] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:37.212257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2f305-5c32-4904-9a94-7acf1a523dfc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:37.325230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2f305-5c32-4904-9a94-7acf1a523dfc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:37.355126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:37.361539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:37.362287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:37.362569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:37.362840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:37.372720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:17:37.377560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:37.380302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:37.381178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 480/940] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:17:37.384878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:37.490299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cc5fa-4725-47d9-9e16-e67ce9709cf3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:37.527562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:37.528329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:37.528639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:37.528997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:37.529760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:37.530084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:37.531448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:37.540820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 461/941] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:37.546817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:37.656200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d42a-1877-48d7-9e89-c6bb0881bc76 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:37.690677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:37.691313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:37.691588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:37.691856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:37.706086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:37.707686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:37.725111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:37.727252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:37.729254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:37.729254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:37.729254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:37.729254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 481/942] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:37.734047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:37.737605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:37.851640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb683e9-93ec-40b5-8687-0fd0eca10294 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:37.893083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:37.893752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:37.894162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:37.894543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:37.895372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:37.896009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:37.897929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:37.911171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 462/943] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:37.917008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:37.966188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:38.031272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdb83d-544c-4b98-9cae-7c78ef9f61f7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:38.067373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:38.068004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:38.068306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:38.068577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:38.075988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:38.077486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:17:38.088578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:38.089646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:38.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 482/944] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:38.095619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:38.165810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:38.209650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34821-d63b-48ee-8107-31120080de3e None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:38.254215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:38.254954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:38.255251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:38.255633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:38.256590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:38.257096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:38.258561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:38.284758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 463/945] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 11 13:17:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:38.286540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:38.404481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d087c1-a4d4-42b5-89cc-371a62816365 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:38.406135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:38.449682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:38.450326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:38.450627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:38.450859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:38.462298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:38.463745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:38.479531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:38.480742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:38.481850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:38.481850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:38.481850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:38.481850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 483/946] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:17:38.487436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:38.609384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821641b-ec1f-4529-b907-def5748f1de7 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:38.637593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:38.659542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:38.660193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:38.660473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:38.660745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:38.661397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:38.661700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:38.663130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:38.676293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 464/947] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:38.682311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:38.788110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa70d6-cae2-4cd7-897d-a955f3107814 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:38.831074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:38.831718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:38.832005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:38.832291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:38.839798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:38.841369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:17:38.853418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:38.854763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:38.855939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 484/948] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:38.860702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:38.971276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebe8e9-3736-4365-bc78-803356c65dc9 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:39.007250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:39.007936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:39.008239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:39.008505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:39.009269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:39.009571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:39.011018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695de91-f7e1-4cb2-9100-a1ad215a7d32 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:39.024832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 465/949] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:39.030452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:39.047255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:39.134438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282339-3bd1-4dd5-aa94-f90be4e17d54 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:39.174905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:39.175696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:39.175969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:39.176336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] PATH_INFO: `/v3/projects/04522cf99e1243c7a8903071b9803075/users/eabb880b598b47c0b7d7f969d4eebffb/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:39.178307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] RBAC: Authorizing `identity:create_grant(project_id=04522cf99e1243c7a8903071b9803075, user_id=eabb880b598b47c0b7d7f969d4eebffb, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:39.179746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf9e78-e25a-4c08-a25a-cf4190b5c317 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:39.194208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 485/950] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 11 13:17:39 2022] PUT /identity/v3/projects/04522cf99e1243c7a8903071b9803075/users/eabb880b598b47c0b7d7f969d4eebffb/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:17:41.451572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa45dd-7372-4d94-b608-2d85af99ccf1 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:41.452144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa45dd-7372-4d94-b608-2d85af99ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:41.452144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa45dd-7372-4d94-b608-2d85af99ccf1 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:41.453867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 466/951] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:41.459230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:41.459682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:41.459771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:41.478219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:41.479387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:41.547390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 486/952] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:41.828527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa45dd-7372-4d94-b608-2d85af99ccf1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:41.876183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:41.935716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa45dd-7372-4d94-b608-2d85af99ccf1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:41.970520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:41.971159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:41.971433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:41.971697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:41.981990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 13:17:41.986837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:41.987854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:41.993302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 467/953] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 11 13:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:41.994653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:41.997852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:42.103899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62abec3-692a-447e-bc8b-97ebec3d52a4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:42.139616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:42.140258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:42.140531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:42.140794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:42.141467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:42.141752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:42.143112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:42.151628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 487/954] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 11 13:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:42.158500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:42.260606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:42.264044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8207-98f5-4013-9e40-e6ed2e38b569 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:42.303747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:42.304692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:42.304989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:42.305344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:42.318798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:42.320726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:42.339360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:42.340837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:42.341900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:42.341900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:42.341900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:42.341900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 468/955] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:42.347682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:42.419305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:42.460845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a9c2-b6db-4ca9-9576-cb884e35ca7e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:42.499954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:42.500600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:42.500877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:42.501148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:42.501795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:42.502126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:42.503509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:42.515819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 488/956] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:42.522068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:42.634033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba6a1-b08b-4c2f-85da-5003538ebc90 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:42.672626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:42.673285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:42.673556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:42.673869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:42.682439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:42.684091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 13:17:42.697763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:42.698756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:42.699721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 469/957] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 13:17:42.705350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:42.813106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c154a9-46b5-4863-a3d3-92bb30eae19e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:42.840515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:42.853285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:42.853940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:42.854214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:42.854483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:42.855230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:42.855540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:42.857023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:42.875139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 489/958] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 11 13:17:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:42.881912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:42.898389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:42.988012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086e069-e041-4cd0-963a-6129f6f5631b None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:43.021188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:43.021762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:43.022035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:43.022306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:43.092511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:43.093903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:43.108236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:43.200868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 470/959] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 11 13:17:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 13:17:43.207317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:43.321602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302d4ce-4086-48d6-9449-4aaec6a6ab63 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:43.359967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:43.360655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:43.360930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:43.361213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:43.372428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:43.373887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 13:17:43.389683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:43.390840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 13:17:43.391740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 13:17:43.391740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 13:17:43.391740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:43.391740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 490/960] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:43.397427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:43.512152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a2f36-60a4-4d30-b0be-72a74852b4d5 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:43.527974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:43.553900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:43.554581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:43.554855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:43.555123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:43.555821 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:43.556199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:43.557643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:43.570961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 471/961] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 11 13:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:43.577132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:43.619252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:43.699612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553272-fb57-48ce-9a08-6a8f73c2d388 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:43.733278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:43.733908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:43.734168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:43.734433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:43.741285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:43.742711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 13:17:43.753805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:43.754873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 11 13:17:43.756001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 491/962] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 11 13:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:43.760982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:43.866581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9ade-5797-43f5-829b-a78c6205d89d None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:43.901468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:43.902077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:43.902390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:43.902684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:43.903455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:43.903785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:43.905379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a576250-05f7-4126-b07f-7658555a982a None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:43.917935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 472/963] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 11 13:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:43.924571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:44.015359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:44.022173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03c79f-9814-4647-a480-b81dabb69aaf None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:44.055518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:44.056141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:44.056415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:44.056682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:44.057408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] RBAC: Adding query filter params (role.id=5ae4cdd2d721472191d8c3e071059abe, scope.project.id=04522cf99e1243c7a8903071b9803075, user.id=eabb880b598b47c0b7d7f969d4eebffb) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:44.057716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:44.059146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e726-cd3b-4e9f-96cf-d284ce2cb759 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:44.064928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 492/964] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 11 13:17:43 2022] GET /identity/v3/role_assignments?user.id=eabb880b598b47c0b7d7f969d4eebffb&scope.project.id=04522cf99e1243c7a8903071b9803075&role.id=5ae4cdd2d721472191d8c3e071059abe => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:46.161433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054bc7-0a14-45ee-baa0-780011e16edd None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:46.162408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054bc7-0a14-45ee-baa0-780011e16edd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:46.162408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054bc7-0a14-45ee-baa0-780011e16edd None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:46.164453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 473/965] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:46.170061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:46.170508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:46.170609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:46.189381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:46.190534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:46.259191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 493/966] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:46.518758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b054bc7-0a14-45ee-baa0-780011e16edd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:46.558212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:46.633353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b054bc7-0a14-45ee-baa0-780011e16edd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:46.668440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:46.669062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:46.669338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:46.669608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:46.670189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:46.671542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:46.680522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:46.681665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:46.682955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 474/967] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:17:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:46.689702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:46.795631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:46.807630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f568756-7e7e-485c-b8e9-78bb1f1d5a1f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:46.845644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:46.846246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:46.846521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:46.846788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:46.847443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:46.847742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:46.849146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e903-0950-405a-ae27-52ac37091717 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:46.854346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 494/968] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 11 13:17:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:46.860967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:46.960945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:46.968057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c670a-2c20-4cc1-9e23-4b7dc7da9885 None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:47.004512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:47.005113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:47.005383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:47.005610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] PATH_INFO: `/v3/services` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:47.006262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:47.006559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:47.007920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda302-103d-43e1-98d2-f5ce23c69f83 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:47.012939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 475/969] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 11 13:17:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:49.086286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d36a2-968b-4466-b7c3-cb0d768bcaf2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:49.087135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d36a2-968b-4466-b7c3-cb0d768bcaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:49.087135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d36a2-968b-4466-b7c3-cb0d768bcaf2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:49.089356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 495/970] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:49.095434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca883e-2453-4e19-8449-930bb4132c01 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:49.095843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca883e-2453-4e19-8449-930bb4132c01 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:49.095997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca883e-2453-4e19-8449-930bb4132c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:49.120887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-22ca883e-2453-4e19-8449-930bb4132c01 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:49.122071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-22ca883e-2453-4e19-8449-930bb4132c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:49.196461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 476/971] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:49.458983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d36a2-968b-4466-b7c3-cb0d768bcaf2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:49.568133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d36a2-968b-4466-b7c3-cb0d768bcaf2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:49.603187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:49.603790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:49.604059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:49.604347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:49.604980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:49.606343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f5cff-419c-446c-9cda-c03f442ef74b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:49.622272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 496/972] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 11 13:17:49 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:17:51.778806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef421-0a38-490d-98be-1d8ed4dc0704 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:51.779418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef421-0a38-490d-98be-1d8ed4dc0704 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:51.779418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef421-0a38-490d-98be-1d8ed4dc0704 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:51.780753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 477/973] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:51.787294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465348-934a-44a0-8220-82061afa741b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:51.788036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465348-934a-44a0-8220-82061afa741b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:51.788201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465348-934a-44a0-8220-82061afa741b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:51.816077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3e465348-934a-44a0-8220-82061afa741b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:51.817287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e465348-934a-44a0-8220-82061afa741b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:51.827566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:51.899660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 497/974] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:52.169462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aef421-0a38-490d-98be-1d8ed4dc0704 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:52.277312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:52.290997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aef421-0a38-490d-98be-1d8ed4dc0704 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:52.329713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:52.330448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:52.330730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:52.330993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] PATH_INFO: `/v3/services/placement` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:52.331578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:52.332971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:52.342029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:52.343126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:52.344042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 478/975] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:17:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:52.349456 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e465348-934a-44a0-8220-82061afa741b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:52.449856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e465348-934a-44a0-8220-82061afa741b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:52.485661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:52.486277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:52.486548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:52.486811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:52.487454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:52.487748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:52.489160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:52.494424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 498/976] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 11 13:17:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:52.500701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:52.603457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90689e-30bd-4f30-89f3-08b9e211f6ba None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:52.638066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:52.638667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:52.638947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:52.639217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:52.639920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] RBAC: Adding query filter params (interface=public, service_id=cf41dbf161d44eafa35de2b18ed3dc02, region_id=RegionOne) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:52.640265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:52.641635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9932f-4c5e-4b30-9af3-211d96900103 None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:52.647056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 479/977] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 11 13:17:52 2022] GET /identity/v3/endpoints?service_id=cf41dbf161d44eafa35de2b18ed3dc02&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:17:52.655884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:52.660810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:52.765793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3d9ac-e57e-4b78-bcbd-b5b461018231 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:52.800908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:52.801520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:52.801771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:52.802061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:52.802643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:52.804013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b24b-0752-4711-ad7c-dfa2c0a9c65d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:52.813293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 499/978] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 11 13:17:52 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:17:54.911441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430c8-cad7-4cc6-a1fb-76fcb7218c15 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:54.912271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430c8-cad7-4cc6-a1fb-76fcb7218c15 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:54.912271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319430c8-cad7-4cc6-a1fb-76fcb7218c15 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:54.914310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 480/979] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:17:54.920518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:54.920948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:54.921081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:54.947341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:17:54.948571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:17:55.024668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:55.030936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 500/980] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:17:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 13:17:55.293632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319430c8-cad7-4cc6-a1fb-76fcb7218c15 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:55.315212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:55.402217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319430c8-cad7-4cc6-a1fb-76fcb7218c15 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:55.439890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:55.440548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:55.440832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:55.441164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:55.441760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:55.443133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 13:17:55.452304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 13:17:55.453435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application with Lock( Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self._enter() Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application return self.creator() Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application created_value = creator( Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: ERROR keystone.server.flask.application Oct 11 13:17:55.454414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 481/981] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 11 13:17:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 13:17:55.459922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:55.564739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079fc7-732b-4a23-994c-55eb7d22fc34 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:55.605316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:55.605929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:55.606208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:55.606482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:55.607119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:17:55.607424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:55.608854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e5c5-e96c-4e1b-85ee-84f18c83ddfb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:55.614133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 501/982] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 11 13:17:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:17:55.619989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:17:55.703723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:17:55.725297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171177b-3c13-4204-b20f-cd9dc3a7b7cb None admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:17:55.760428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:17:55.761019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:17:55.761297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:17:55.761565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:17:55.762230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:17:55.763598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535721-9b01-455d-b951-5ab562bc526c None admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:17:55.786547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 482/983] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:17:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:18:05.610911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=69 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:18:05.782607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:18:58.346641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453608-f6ef-4e7e-a461-8927965ea81d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:18:58.347745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453608-f6ef-4e7e-a461-8927965ea81d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:18:58.348272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453608-f6ef-4e7e-a461-8927965ea81d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:18:58.355962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aec97-75c9-4859-b315-2f3c29cff2c6 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:18:58.357012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aec97-75c9-4859-b315-2f3c29cff2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:18:58.357499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aec97-75c9-4859-b315-2f3c29cff2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:18:58.360609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 502/984] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:18:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:18:58.571871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d24aec97-75c9-4859-b315-2f3c29cff2c6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:18:58.573127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d24aec97-75c9-4859-b315-2f3c29cff2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:18:58.730239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 483/985] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:18:59.022491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:18:59.023024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:18:59.023487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:18:59.024901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 503/986] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 11 13:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:18:59.031721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074b06-bb5c-4102-8247-40cc9aeb7e4c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:18:59.032327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074b06-bb5c-4102-8247-40cc9aeb7e4c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:18:59.032578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074b06-bb5c-4102-8247-40cc9aeb7e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:18:59.123724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-db074b06-bb5c-4102-8247-40cc9aeb7e4c None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:18:59.125146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-db074b06-bb5c-4102-8247-40cc9aeb7e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:18:59.148736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:18:59.209284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 484/987] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 11 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:18:59.218525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:18:59.220355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:18:59.333042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8833a5-0c19-4d5c-8315-d8c6ca7d0b85 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:18:59.410166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:18:59.410794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:18:59.411033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:18:59.411221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:18:59.412968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:18:59.479080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:18:59.540879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:18:59.542440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e428e-89c9-477e-998e-30279e4278d0 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:18:59.562452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 504/988] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:01.966128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:01.966704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:01.966704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:01.968078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 485/989] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:01.974448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cec18-c0e4-4cc3-92bb-9ee60633fe0d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:01.975131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cec18-c0e4-4cc3-92bb-9ee60633fe0d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:01.975292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cec18-c0e4-4cc3-92bb-9ee60633fe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:02.057061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e21cec18-c0e4-4cc3-92bb-9ee60633fe0d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:02.058291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e21cec18-c0e4-4cc3-92bb-9ee60633fe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:02.162208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 505/990] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:02.442887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:02.444528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:02.561231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965b623-b88a-4af1-8cc4-65690f1cdf35 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:02.641302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:02.643752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:02.644717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:02.644997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:02.645266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:02.647425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:02.757812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:02.759123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21960a0-3c9b-40ab-a9cd-bfd96c121a45 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:02.778136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 486/991] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:19:05.180327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd27e-1bef-4773-b270-fa1f15da660d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:05.180865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd27e-1bef-4773-b270-fa1f15da660d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:05.180865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd27e-1bef-4773-b270-fa1f15da660d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:05.182200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 506/992] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:05.188798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d63e4a-1751-4fb6-8c4c-ab7bbc4e194d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:05.189536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d63e4a-1751-4fb6-8c4c-ab7bbc4e194d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:05.189672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d63e4a-1751-4fb6-8c4c-ab7bbc4e194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:05.273529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d9d63e4a-1751-4fb6-8c4c-ab7bbc4e194d None None] MFA Rules not processed for user `394c672b73ca4f16aca0a84b753bfd89`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:05.274749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d9d63e4a-1751-4fb6-8c4c-ab7bbc4e194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:05.328982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:05.374309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 487/993] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:05.678297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:05.678584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:05.678844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:05.680248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 507/994] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 11 13:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:05.686840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c612d-0dc5-4d3d-a977-52a22577deb9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:05.687317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c612d-0dc5-4d3d-a977-52a22577deb9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:05.687782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c612d-0dc5-4d3d-a977-52a22577deb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:05.772778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5f9c612d-0dc5-4d3d-a977-52a22577deb9 None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:05.774000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5f9c612d-0dc5-4d3d-a977-52a22577deb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:05.851094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 488/995] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 11 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:05.860273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:05.861801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:05.872399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:05.972634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43018143-28ef-4447-9ff7-7c57fb628212 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:06.059272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:06.060193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:06.060682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:06.061103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:06.062857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:06.126902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:06.185894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:06.187452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1f4f-ce57-4ae4-978c-23686f75a1fc service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:06.206060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 508/996] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:09.231864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:09.232804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:09.232804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:09.234347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 489/997] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:09.240002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbce73-7e29-4a72-bd23-30e6e555963c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:09.240455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbce73-7e29-4a72-bd23-30e6e555963c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:09.240591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbce73-7e29-4a72-bd23-30e6e555963c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:09.315846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-49bbce73-7e29-4a72-bd23-30e6e555963c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:09.317100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-49bbce73-7e29-4a72-bd23-30e6e555963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:09.414430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 509/998] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:09.696787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:09.699298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:09.759278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:09.816387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c625b7-960f-46c4-aad4-0883edf7addf None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:09.892543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:09.893074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:09.893349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:09.893616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:09.895534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:09.996094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:10.011303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:10.012774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1c44-a645-456d-bc9e-a578e8f767cd service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:10.032466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 490/999] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:12.852894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:12.853421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:12.853421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:12.854653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 510/1000] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:12.860276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba2a8f-73db-4773-9fac-13b84039d053 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:12.860731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba2a8f-73db-4773-9fac-13b84039d053 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:12.860933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba2a8f-73db-4773-9fac-13b84039d053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:12.934559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f3ba2a8f-73db-4773-9fac-13b84039d053 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:12.936044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f3ba2a8f-73db-4773-9fac-13b84039d053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:13.040989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 491/1001] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:19:13.317454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:13.318977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:13.339869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:13.427576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475f59b-c4aa-4e6c-9df5-d1bf8721217a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:13.503309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:13.503926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:13.504233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:13.504498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:13.506097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:13.583120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:13.621941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:13.623399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b63f2-d45c-463e-9382-58694142f086 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:13.643266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 511/1002] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:15.827171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:15.827751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:15.827751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:15.829093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 492/1003] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:15.834478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde25dd5-6721-4767-9b37-7db3261cba9e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:15.834912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde25dd5-6721-4767-9b37-7db3261cba9e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:15.835010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde25dd5-6721-4767-9b37-7db3261cba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:15.902138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dde25dd5-6721-4767-9b37-7db3261cba9e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:15.903243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dde25dd5-6721-4767-9b37-7db3261cba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:16.010128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 512/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:16.283772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:16.285435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:16.349530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:16.393861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5e357-8874-4701-b01f-f29a21df4199 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:16.466115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:16.466727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:16.466999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:16.467321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:16.468955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:16.582641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:16.585415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:16.587816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab246d0-930f-49b4-9052-10d3e7f6ad58 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:16.606552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 493/1005] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:18.764916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:18.765441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:18.765441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:18.766725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 513/1006] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:18.773471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfb7f-e01f-4970-893f-9c55edfd7148 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:18.774177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfb7f-e01f-4970-893f-9c55edfd7148 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:18.774355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfb7f-e01f-4970-893f-9c55edfd7148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:18.846315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-62abfb7f-e01f-4970-893f-9c55edfd7148 None None] MFA Rules not processed for user `394c672b73ca4f16aca0a84b753bfd89`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:18.847548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-62abfb7f-e01f-4970-893f-9c55edfd7148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:18.950300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 494/1007] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:19:19.228771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:19.230331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:19.265218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:19.344193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f3896-bd01-4fa1-8c79-f33856cd6c9c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:19.418088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:19.418626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:19.418902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:19.419170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:19.420820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:19.499898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:19.533352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:19.534793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01978bbc-4791-47a9-bde3-cfd0051b46f7 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:19.553008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 514/1008] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:21.907593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:21.908188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:21.908188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:21.909511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 495/1009] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:21.915856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee260cba-da20-4ec0-834c-162c99152ea3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:21.916615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee260cba-da20-4ec0-834c-162c99152ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:21.916763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee260cba-da20-4ec0-834c-162c99152ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:21.993614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ee260cba-da20-4ec0-834c-162c99152ea3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:21.994835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ee260cba-da20-4ec0-834c-162c99152ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:22.099931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 515/1010] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:22.388340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:22.390819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:22.465649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:22.501241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0eca-d7e4-4665-a39d-ba72c9e9e7b6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:22.579176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:22.579787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:22.580071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:22.580359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:22.581959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:22.700471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:22.702206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6701-3acc-4141-a2f3-377c369700c7 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:22.713011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:22.721806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 496/1011] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:24.875996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:24.876586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:24.876586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:24.878178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 516/1012] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:24.885836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77a4f-317e-4cfb-9da1-c90a2cd7a10e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:24.886337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77a4f-317e-4cfb-9da1-c90a2cd7a10e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:24.886337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77a4f-317e-4cfb-9da1-c90a2cd7a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:24.955362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d3e77a4f-317e-4cfb-9da1-c90a2cd7a10e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:24.957191 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d3e77a4f-317e-4cfb-9da1-c90a2cd7a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:25.067810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 497/1013] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:19:25.351632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:25.353224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:25.398615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:25.464509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a71ac-dff3-4f56-b012-95a86a149429 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:25.542871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:25.543484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:25.543763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:25.544033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:25.545733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:25.646892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:25.676355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:25.677799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0294-9bc8-425e-9123-b57ef0b1e0d3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:25.696783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 517/1014] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:28.118295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:28.119149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:28.119356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:28.121524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 498/1015] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:28.129761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd8fe-4084-4312-aa1e-5ff6d80454f5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:28.130212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd8fe-4084-4312-aa1e-5ff6d80454f5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:28.130312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd8fe-4084-4312-aa1e-5ff6d80454f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:28.196600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f53fd8fe-4084-4312-aa1e-5ff6d80454f5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:28.197796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f53fd8fe-4084-4312-aa1e-5ff6d80454f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:28.298560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 518/1016] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:19:28.574531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:28.577041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:28.660551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:28.691564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92966f3c-df9b-49d2-957e-0eb681aaf6f1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:28.768214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:28.769134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:28.769417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:28.769676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:28.771269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:28.884970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:28.886294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdb8a5-1838-4635-b939-3b139f32b496 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:28.906421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 499/1017] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:33.035628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:33.036233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:33.036233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:33.037547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 519/1018] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:33.042925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462724f1-6f72-47f1-8488-18492df97c95 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:33.043327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462724f1-6f72-47f1-8488-18492df97c95 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:33.043490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462724f1-6f72-47f1-8488-18492df97c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:33.049151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:33.113833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-462724f1-6f72-47f1-8488-18492df97c95 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:33.114947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-462724f1-6f72-47f1-8488-18492df97c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:33.214661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 500/1019] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:33.493735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:33.495267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:33.548560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:33.605149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bffb1e-0c60-452f-b22f-873f1f263d69 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:33.679659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:33.680451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:33.680722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:33.680995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:33.682625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:33.779677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:33.797413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:33.798851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea55f17-a41c-42ab-bcfe-47a40f571473 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:33.818091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 520/1020] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:36.561297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:36.561880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:36.561880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:36.563365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 501/1021] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:36.568888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82830f-f6b9-4ade-a9df-cd369aca69b2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:36.569277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82830f-f6b9-4ade-a9df-cd369aca69b2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:36.569455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82830f-f6b9-4ade-a9df-cd369aca69b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:36.639026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2c82830f-f6b9-4ade-a9df-cd369aca69b2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:36.640240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c82830f-f6b9-4ade-a9df-cd369aca69b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:36.740825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 521/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:37.024901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:37.026460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:37.123911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:37.142086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a596d-d262-4b42-ac86-915f111009cd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:37.220095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:37.220666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:37.220944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:37.221215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:37.222836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:37.337392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:37.338828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d174a5-898a-4f77-82e9-baf131d567d6 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:37.357547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 502/1023] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:41.654476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:41.654935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:41.655027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:41.656542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 522/1024] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:41.662153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a960dd8-7d77-4725-a114-7f19ffb37bc3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:41.662553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a960dd8-7d77-4725-a114-7f19ffb37bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:41.662733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a960dd8-7d77-4725-a114-7f19ffb37bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:41.680877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:41.741425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-3a960dd8-7d77-4725-a114-7f19ffb37bc3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:41.742768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3a960dd8-7d77-4725-a114-7f19ffb37bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:41.848091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 503/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:42.131558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:42.134007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:42.200819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:42.249500 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6a8e-0401-453b-ba12-b31fe948d722 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:42.335636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:42.336268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:42.336545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:42.336811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:42.338405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:42.446509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:42.450632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:42.452278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48a717-1ff6-4204-9bb0-05a1fff9100f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:42.471383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 523/1026] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:19:47.265713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f66f5-bbcc-44b9-a815-b2668660d280 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:47.266276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f66f5-bbcc-44b9-a815-b2668660d280 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:47.266494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f66f5-bbcc-44b9-a815-b2668660d280 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:47.267910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 504/1027] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:47.273546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205dcbf-9259-4dab-b404-2407307c88c1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:47.273950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205dcbf-9259-4dab-b404-2407307c88c1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:47.274086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205dcbf-9259-4dab-b404-2407307c88c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:47.376002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1205dcbf-9259-4dab-b404-2407307c88c1 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:47.377291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1205dcbf-9259-4dab-b404-2407307c88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:47.552774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 524/1028] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:19:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:51.573174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de022c-59de-4400-90e9-0f6e207bd981 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:51.573174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de022c-59de-4400-90e9-0f6e207bd981 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:51.573174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de022c-59de-4400-90e9-0f6e207bd981 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:51.574428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 505/1029] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:19:51.579568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155415f-864c-4b99-bceb-3d7914adb753 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:51.580024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155415f-864c-4b99-bceb-3d7914adb753 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:51.580312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155415f-864c-4b99-bceb-3d7914adb753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:51.641115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:51.668875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b155415f-864c-4b99-bceb-3d7914adb753 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:51.670158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b155415f-864c-4b99-bceb-3d7914adb753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:51.781460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 525/1030] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:19:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:51.846543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=59 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:53.113669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234eced8-b069-4b49-b4c6-746711dd292a None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:53.114312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234eced8-b069-4b49-b4c6-746711dd292a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:53.114312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234eced8-b069-4b49-b4c6-746711dd292a None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:53.115536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 506/1031] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:53.120719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed501e-e047-48ea-8f62-39fd73755e48 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:53.121122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed501e-e047-48ea-8f62-39fd73755e48 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:53.121286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed501e-e047-48ea-8f62-39fd73755e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:53.203025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-aeed501e-e047-48ea-8f62-39fd73755e48 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:53.204270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aeed501e-e047-48ea-8f62-39fd73755e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:53.248132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:53.314628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 526/1032] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:19:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:56.883778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23034-5c65-40e3-a9c5-b211c28f50ff None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:56.884338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23034-5c65-40e3-a9c5-b211c28f50ff None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:56.884338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23034-5c65-40e3-a9c5-b211c28f50ff None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:56.885585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 507/1033] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:56.891169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699224a5-c2c4-453a-8ad6-9ea757f50ef6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:56.891657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699224a5-c2c4-453a-8ad6-9ea757f50ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:56.891746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699224a5-c2c4-453a-8ad6-9ea757f50ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:56.968105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-699224a5-c2c4-453a-8ad6-9ea757f50ef6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:56.969370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-699224a5-c2c4-453a-8ad6-9ea757f50ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:57.077208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 527/1034] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:57.374747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:57.375361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:57.375999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:57.378322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 508/1035] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:19:57.388171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761c7-2b87-4e6f-a3e1-cb234d4d14bc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:57.388778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761c7-2b87-4e6f-a3e1-cb234d4d14bc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:57.389867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839761c7-2b87-4e6f-a3e1-cb234d4d14bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:57.410440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:57.479930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-839761c7-2b87-4e6f-a3e1-cb234d4d14bc None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:19:57.481636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-839761c7-2b87-4e6f-a3e1-cb234d4d14bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:57.619529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 528/1036] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:19:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:19:57.628177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:19:57.629868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:57.798157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bb55b-2e2d-4d98-bb88-aabcd743cd14 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:19:57.876465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:19:57.919547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:19:57.920259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:19:57.920476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:19:57.920704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:19:57.922310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:19:58.036226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:19:58.037796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23ab7-0bac-409d-b5e6-a47177b8b559 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:19:58.058893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 509/1037] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:00.432315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:00.432315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:00.433843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:00.436731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 529/1038] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:00.439439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e7b9-1cd1-4dd7-b94a-7f589bd4d8cd None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:00.439439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e7b9-1cd1-4dd7-b94a-7f589bd4d8cd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:00.440241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e7b9-1cd1-4dd7-b94a-7f589bd4d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:00.505939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bbd2e7b9-1cd1-4dd7-b94a-7f589bd4d8cd None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:00.507164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bbd2e7b9-1cd1-4dd7-b94a-7f589bd4d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:00.549775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:00.612071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 510/1039] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:00.891796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:00.893595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:00.971225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:01.080719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585cab8-8866-43aa-b234-3b47aa024af3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:01.223924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:01.224635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:01.224909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:01.225170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:01.226733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:01.249976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:01.318102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:01.319618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cec35-9de3-4630-96f2-12def4df2d06 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:01.337707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 530/1040] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:03.795240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb5dde-ba8f-44c8-9120-df39a6a26805 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:03.795838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb5dde-ba8f-44c8-9120-df39a6a26805 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:03.795838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb5dde-ba8f-44c8-9120-df39a6a26805 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:03.797031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 511/1041] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:03.802854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a76f-2579-4255-8f5d-0245e39b9d22 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:03.803294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a76f-2579-4255-8f5d-0245e39b9d22 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:03.803411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a76f-2579-4255-8f5d-0245e39b9d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:03.871312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2416a76f-2579-4255-8f5d-0245e39b9d22 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:03.872622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2416a76f-2579-4255-8f5d-0245e39b9d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:03.967872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:03.972163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 531/1042] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:04.253957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:04.254238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:04.254509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:04.256099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 512/1043] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:04.261798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d19d6-3bfb-40c2-bc88-113fc19b701d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:04.262183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d19d6-3bfb-40c2-bc88-113fc19b701d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:04.262490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d19d6-3bfb-40c2-bc88-113fc19b701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:04.344574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3b9d19d6-3bfb-40c2-bc88-113fc19b701d None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:04.345784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3b9d19d6-3bfb-40c2-bc88-113fc19b701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:04.464236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 532/1044] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:04.472270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:04.473774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:04.629358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1f6c2-ad06-456c-86b7-1826190cf28e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:04.651952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:04.756886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:04.757680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:04.758006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:04.758276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:04.760036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:04.860968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:04.862678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2163f39-e5af-4837-a67b-b99f3fd9d528 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:04.886460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 513/1045] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:04.980541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a04c-316e-4454-94d6-6166c81e1fa5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:04.981208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a04c-316e-4454-94d6-6166c81e1fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:04.981498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a04c-316e-4454-94d6-6166c81e1fa5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:04.983422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 533/1046] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:04.988654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4293-37ff-4545-a0fe-38649cc02ce1 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:04.988938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4293-37ff-4545-a0fe-38649cc02ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:04.989257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d4293-37ff-4545-a0fe-38649cc02ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:05.028934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:05.072277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-4a1d4293-37ff-4545-a0fe-38649cc02ce1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:05.073520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4a1d4293-37ff-4545-a0fe-38649cc02ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:05.191513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 514/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:05.522331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:05.522331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:05.522331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:05.522331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 534/1048] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 11 13:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:05.527341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123494b-37f2-4a07-a098-f5c8988cb837 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:05.527640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123494b-37f2-4a07-a098-f5c8988cb837 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:05.527940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123494b-37f2-4a07-a098-f5c8988cb837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:05.615396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a123494b-37f2-4a07-a098-f5c8988cb837 None None] MFA Rules not processed for user `3ecc3d37e6214164a01eb96629509215`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:05.616848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a123494b-37f2-4a07-a098-f5c8988cb837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:05.673094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:05.700722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 515/1049] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 11 13:20:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:20:05.708607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:05.710184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:05.761611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:05.820477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5549-4548-4635-b0e2-7b14b6c7a008 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:05.907237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:05.907237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:05.907237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:05.907237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:05.907794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:06.012605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:06.030221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:06.031948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac4d2-114a-4669-a597-3572af625d76 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:06.054806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 535/1050] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:20:07.096053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:07.096642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:07.096642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:07.097997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 516/1051] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:07.103423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fdfc8-3107-4c9b-bcd7-2737625426c6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:07.103900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fdfc8-3107-4c9b-bcd7-2737625426c6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:07.104002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fdfc8-3107-4c9b-bcd7-2737625426c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:07.171914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dd4fdfc8-3107-4c9b-bcd7-2737625426c6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:07.173176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dd4fdfc8-3107-4c9b-bcd7-2737625426c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:07.268872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 536/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:07.562161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:07.564018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:07.728911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10034bc-3a88-421b-bd8a-6d6b4a4eaed2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:07.781802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:07.845782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:07.846399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:07.846676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:07.846911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:07.848556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:07.939788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:07.941388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f878bb-6e83-460c-9227-2cf3832d8f00 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:07.959575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 517/1053] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:20:10.165388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:10.165933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:10.165933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:10.167454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 537/1054] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:10.172628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631b65-1356-4816-b5a4-a527f7d81d0a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:10.173098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631b65-1356-4816-b5a4-a527f7d81d0a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:10.173179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631b65-1356-4816-b5a4-a527f7d81d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:10.240564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ed631b65-1356-4816-b5a4-a527f7d81d0a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:10.241743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ed631b65-1356-4816-b5a4-a527f7d81d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:10.249407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:10.348640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 518/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:10.636183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:10.637799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:10.706613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:10.797370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8441c4-b562-4803-a1fa-932d2bd9be71 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:10.915771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:10.916351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:10.916625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:10.917008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:10.918713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:10.938854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:11.011683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:11.013387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22695-2cb1-4e35-81d8-8a71e43349e5 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:11.034090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 538/1056] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:13.190896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:13.191514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:13.191514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:13.193139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 519/1057] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:13.198287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5cc3-862c-4838-89b9-3a61c26ef64b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:13.198707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5cc3-862c-4838-89b9-3a61c26ef64b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:13.198821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5cc3-862c-4838-89b9-3a61c26ef64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:13.268835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-633f5cc3-862c-4838-89b9-3a61c26ef64b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:13.270087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-633f5cc3-862c-4838-89b9-3a61c26ef64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:13.378800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:13.392908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 539/1058] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:20:13.667990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:13.669988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:13.838547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:13.856279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec30e5-fbf9-4ffc-b3dc-6852e90ab757 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:13.983658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:13.984318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:13.984615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:13.984858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:13.986462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:14.098848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:14.100803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdc776-77ef-49e0-b1a7-3798af2e16d8 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:14.113337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:14.124804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 520/1059] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:16.248051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:16.248778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:16.248778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:16.249953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 540/1060] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:16.255646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f171-4519-46c7-bf9b-7710afb61137 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:16.256084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f171-4519-46c7-bf9b-7710afb61137 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:16.256451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f171-4519-46c7-bf9b-7710afb61137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:16.341145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f0e5f171-4519-46c7-bf9b-7710afb61137 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:16.342883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f0e5f171-4519-46c7-bf9b-7710afb61137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:16.454238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 521/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:16.735628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:16.737218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:16.884771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fa62d-7ab1-4f13-879b-4323f7f66463 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:16.955391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:17.000609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:17.001220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:17.001490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:17.001748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:17.003314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:17.094707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:17.096336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660fc0a-00c4-486d-a4f5-9d15040d362c service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:17.113858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 541/1062] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:19.393637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:19.394252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:19.394252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:19.395567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 522/1063] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:19.401263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bb824-e768-4d0d-8c85-4511ea6b2306 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:19.401712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bb824-e768-4d0d-8c85-4511ea6b2306 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:19.401810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bb824-e768-4d0d-8c85-4511ea6b2306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:19.471359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ae0bb824-e768-4d0d-8c85-4511ea6b2306 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:19.472600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ae0bb824-e768-4d0d-8c85-4511ea6b2306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:19.505962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:19.575981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 542/1064] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:19.865466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:19.867083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:19.954490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:20.025667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1648d-5679-44cd-ae7c-ef7cd868a345 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:20.139376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:20.139986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:20.140291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:20.140560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:20.142148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:20.183350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:20.235154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:20.236679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b10c2-2ddd-4bbb-986d-d93acd9c7978 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:20.254972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 523/1065] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:21.197645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bb95f-e037-49e8-958a-bb6b14ae1d19 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:21.198227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bb95f-e037-49e8-958a-bb6b14ae1d19 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:21.198227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bb95f-e037-49e8-958a-bb6b14ae1d19 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:21.199648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 543/1066] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:21.205173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9d01-7a68-47a3-ac42-5e1def5f17b9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:21.205569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9d01-7a68-47a3-ac42-5e1def5f17b9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:21.205750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9d01-7a68-47a3-ac42-5e1def5f17b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:21.273044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ef6c9d01-7a68-47a3-ac42-5e1def5f17b9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:21.274281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ef6c9d01-7a68-47a3-ac42-5e1def5f17b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:21.372698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 524/1067] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:21.664557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:21.664837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:21.665103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:21.666500 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 544/1068] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 11 13:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:21.672573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347522-a3b7-4424-bb8a-c7201905b7dc None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:21.672882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347522-a3b7-4424-bb8a-c7201905b7dc None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:21.673156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347522-a3b7-4424-bb8a-c7201905b7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:21.690563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:21.751566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-60347522-a3b7-4424-bb8a-c7201905b7dc None None] MFA Rules not processed for user `f056f1106e9a4b5ba29fe99058e47c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:21.752776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-60347522-a3b7-4424-bb8a-c7201905b7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:21.828624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 525/1069] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 11 13:20:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:21.836716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:21.838252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:21.942883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad1c4-97f3-49f2-abd5-8373101eaf35 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:22.004134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:22.020073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:22.020790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:22.021067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:22.021331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:22.022896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:22.135187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:22.136705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ffbff-c25c-4ba8-a48e-db017f1dff50 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:22.155742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 545/1070] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:22.435798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342d69-83bf-43c6-8a7a-5c39d3d6dbf4 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:22.436452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342d69-83bf-43c6-8a7a-5c39d3d6dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:22.436452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342d69-83bf-43c6-8a7a-5c39d3d6dbf4 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:22.437731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 526/1071] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:22.443513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:22.443947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:22.444045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:22.523116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:22.524377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:22.578123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:22.635428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 546/1072] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:20:22.892357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1819-e17b-4a39-b57d-3db6c28c3d62 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:22.892630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1819-e17b-4a39-b57d-3db6c28c3d62 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:22.893220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1819-e17b-4a39-b57d-3db6c28c3d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:22.928326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:22.929575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:22.961819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a00c1819-e17b-4a39-b57d-3db6c28c3d62 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:22.963012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a00c1819-e17b-4a39-b57d-3db6c28c3d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:23.018810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:23.052012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 527/1073] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:23.067492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbbb49-7991-4a45-b64e-e53eea1374e7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:23.097294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d4eb-79cf-4462-959f-70b31389e7b6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:23.097577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d4eb-79cf-4462-959f-70b31389e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:23.097851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d4eb-79cf-4462-959f-70b31389e7b6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:23.099201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 528/1074] 10.222.0.15 () {54 vars in 943 bytes} [Tue Oct 11 13:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:20:23.105274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:23.107472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:23.108196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:23.126925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:23.188189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:23.188815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:23.189092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:23.189360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:23.190964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] MFA Rules not processed for user `8f89a9124ef147f9a402e7df844ed088`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:23.191089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:23.192202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:23.282279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:23.284153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555660-d7c1-4887-9951-fe7b06e7c86a service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:23.299301 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 529/1075] 10.222.0.15 () {58 vars in 1040 bytes} [Tue Oct 11 13:20:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:23.304485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 547/1076] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:23.308555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:23.317980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519508fd-69d2-4146-aed5-090bf9e4966b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:23.343682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:23.437510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:23.438283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:23.438407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:23.438647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:23.440340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:23.534036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:23.535634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abbec1-e6fe-4889-941a-4592702df381 service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:23.544466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 530/1077] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:20:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:20:25.479168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:25.479736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:25.479736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:25.481189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 548/1078] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:25.486274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd2a2-f6d8-44fa-b995-45bf58d02904 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:25.486766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd2a2-f6d8-44fa-b995-45bf58d02904 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:25.487112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd2a2-f6d8-44fa-b995-45bf58d02904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:25.563099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-4c4bd2a2-f6d8-44fa-b995-45bf58d02904 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:25.564626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4c4bd2a2-f6d8-44fa-b995-45bf58d02904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:25.576029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:25.671019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 531/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:20:25.955415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:25.957051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:26.027418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:26.118341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d59b2d-2557-4407-b4f7-418b365d7951 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:26.239876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:26.240802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:26.241074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:26.241338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:26.242917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:26.274675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:26.340505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:26.342102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf6f0-d445-4e9a-a60a-70c596ee8ad4 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:26.366940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 549/1080] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:28.535437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:28.536034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:28.536034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:28.537516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 532/1081] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:28.542936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2002e4-3f87-479b-8dd4-e62b4fbd285f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:28.543423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2002e4-3f87-479b-8dd4-e62b4fbd285f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:28.543529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2002e4-3f87-479b-8dd4-e62b4fbd285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:28.617555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ab2002e4-3f87-479b-8dd4-e62b4fbd285f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:28.618927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ab2002e4-3f87-479b-8dd4-e62b4fbd285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:28.732306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 550/1082] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:29.018551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:29.020100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:29.169937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:29.181683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cdfee-2c77-4282-820c-8126dffb0f01 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:29.302831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:29.303373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:29.303646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:29.303944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:29.305744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:29.405831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:29.407577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8645d2-eeb6-4bbc-96d4-d50da732dd14 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:29.418074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:29.427609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 533/1083] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:31.584210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:31.584773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:31.584773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:31.588698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 551/1084] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:31.591784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af1b8-2cca-4520-8343-5a739b5cd622 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:31.592194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af1b8-2cca-4520-8343-5a739b5cd622 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:31.592365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af1b8-2cca-4520-8343-5a739b5cd622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:31.665059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0a5af1b8-2cca-4520-8343-5a739b5cd622 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:31.666292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0a5af1b8-2cca-4520-8343-5a739b5cd622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:31.777312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 534/1085] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:32.057479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:32.059083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:32.062086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:32.224937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53bf13-6e85-4bde-bd70-b3c3c9f7268f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:32.307636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:32.348425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:32.348425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:32.348425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:32.348425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:32.352926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:32.444835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:32.446535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029e6a6-f82c-4b76-a127-e4c1bbd78ded service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:32.465660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 552/1086] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:41.775752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=67 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:42.462095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=61 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:46.003775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d006ce-f879-41b5-97c2-fec111242d97 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:46.004410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d006ce-f879-41b5-97c2-fec111242d97 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:46.004410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d006ce-f879-41b5-97c2-fec111242d97 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:46.006253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 535/1087] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 11 13:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:46.011600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce87392-5a7d-4adc-b0d3-dfc0d6a7b41d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:46.012045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce87392-5a7d-4adc-b0d3-dfc0d6a7b41d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:46.012168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce87392-5a7d-4adc-b0d3-dfc0d6a7b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:46.091091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6ce87392-5a7d-4adc-b0d3-dfc0d6a7b41d None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:46.092319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6ce87392-5a7d-4adc-b0d3-dfc0d6a7b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:46.203849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 553/1088] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 11 13:20:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:46.218564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dfea-c384-49e6-962d-1858a1aa3594 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:46.218842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dfea-c384-49e6-962d-1858a1aa3594 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:46.219111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dfea-c384-49e6-962d-1858a1aa3594 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:46.220492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 536/1089] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 11 13:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:46.225472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900604-da50-40be-8b3c-800c41825550 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:46.225755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900604-da50-40be-8b3c-800c41825550 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:46.226020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900604-da50-40be-8b3c-800c41825550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:46.304329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:46.307511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-45900604-da50-40be-8b3c-800c41825550 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:46.309610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45900604-da50-40be-8b3c-800c41825550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:46.433524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 554/1090] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 11 13:20:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:48.989776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:48.990391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:48.990391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:48.991672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 537/1091] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:48.998182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30e7f-79f2-4c22-bc71-173b7664832f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:48.998911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30e7f-79f2-4c22-bc71-173b7664832f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:48.999086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30e7f-79f2-4c22-bc71-173b7664832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:49.085087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0ba30e7f-79f2-4c22-bc71-173b7664832f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:49.086291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ba30e7f-79f2-4c22-bc71-173b7664832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:49.143708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:49.187869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 555/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:49.476393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:49.478288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:49.633135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc78c5-ebe4-4695-982c-27dd79f6c4c7 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:49.712843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:49.755426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:49.756042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:49.756355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:49.756623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:49.758232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:49.860227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:49.861878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfa0e6-1a7f-4c11-861b-ddb007057fac service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:49.881371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 538/1093] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:51.007141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:51.008071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:51.008071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:51.010273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 556/1094] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:51.021274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38691f6-9559-4024-b18a-f604980416e0 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:51.021734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38691f6-9559-4024-b18a-f604980416e0 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:51.021855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38691f6-9559-4024-b18a-f604980416e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:51.102206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d38691f6-9559-4024-b18a-f604980416e0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:51.103411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d38691f6-9559-4024-b18a-f604980416e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:51.131150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:51.203186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 539/1095] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:51.357122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:51.359418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:51.516663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3a31-16c6-4ad2-b32f-4b52ff21cd70 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:51.563194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:51.649549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:51.650315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:51.650671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:51.651224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:51.653221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:51.744830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:51.746429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e2a39-1b9e-41c4-84f3-145b5a9921ba service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:51.765395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 557/1096] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:52.895371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:52.895923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:52.895923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:52.897915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 540/1097] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:52.905193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830798-2392-4afe-83c7-ec70f2e9e622 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:52.905670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830798-2392-4afe-83c7-ec70f2e9e622 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:52.905776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830798-2392-4afe-83c7-ec70f2e9e622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:52.963924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:52.987436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-19830798-2392-4afe-83c7-ec70f2e9e622 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:52.988685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-19830798-2392-4afe-83c7-ec70f2e9e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:53.086989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 558/1098] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:53.241891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:53.244286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:53.409645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd928e15-a76f-4555-85c0-b186d4182b58 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:53.423524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:53.531486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:53.532014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:53.532316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:53.532584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:53.534171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:53.635788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:53.637409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398801-80a8-4471-98dc-fde5ed007fdc service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:53.657078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 541/1099] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:54.761090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:54.761980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:54.761980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:54.763963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 559/1100] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:20:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:54.771362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b29e-2720-4678-bde4-23d3ea09a219 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:54.771861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b29e-2720-4678-bde4-23d3ea09a219 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:54.771861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53b29e-2720-4678-bde4-23d3ea09a219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:54.794016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:54.851097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5e53b29e-2720-4678-bde4-23d3ea09a219 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:54.852253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5e53b29e-2720-4678-bde4-23d3ea09a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:54.954810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 542/1101] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:55.109648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:55.111161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:55.225918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:55.261667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811009-62b4-4f20-9665-f460670d5077 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:55.387064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:55.387664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:55.387922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:55.388217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:55.389810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:55.476829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:55.491406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:55.493032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3315c-4e28-446b-ab0b-cf3bbb13c597 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:55.511576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 560/1102] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:56.611746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:56.612335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:56.612335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:56.613676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 543/1103] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:20:56.620254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d932-a251-44fa-842c-35f1c9765c45 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:56.620695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d932-a251-44fa-842c-35f1c9765c45 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:56.620817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d932-a251-44fa-842c-35f1c9765c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:56.700885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6eb4d932-a251-44fa-842c-35f1c9765c45 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:56.702089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6eb4d932-a251-44fa-842c-35f1c9765c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:56.805408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 561/1104] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:56.957147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:56.958682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:57.048968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:57.114039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defd0bf-8b55-441e-ba0b-b60e5ff04552 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:57.229307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:57.229918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:57.230188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:57.230451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:57.232052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:57.285399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:57.330408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:57.332001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18d58b-ee39-48d2-8b38-0310e4693321 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:57.350081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 544/1105] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:20:58.394975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:58.395871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:58.395871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:58.398069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 562/1106] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:20:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 13:20:58.404447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00b0a7-2740-406f-88e7-53060eb03591 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:58.404910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00b0a7-2740-406f-88e7-53060eb03591 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:58.405015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00b0a7-2740-406f-88e7-53060eb03591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:58.482203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9b00b0a7-2740-406f-88e7-53060eb03591 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:20:58.483367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9b00b0a7-2740-406f-88e7-53060eb03591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:58.586377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 545/1107] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:20:58.751727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:20:58.753320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:58.817497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:58.931059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdd902-3443-402c-9400-e47f4b02692e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:20:59.063217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:20:59.063874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:20:59.064170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:20:59.064464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:20:59.066981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:20:59.076452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:20:59.169399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:20:59.170988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d323f61-01a2-4acf-a567-d771c4c0b1f7 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:20:59.189432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 563/1108] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:00.262995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:00.263619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:00.263619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:00.264936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 546/1109] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:00.272580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1120c-c134-46dc-af5a-7360420246a3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:00.273363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1120c-c134-46dc-af5a-7360420246a3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:00.273544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1120c-c134-46dc-af5a-7360420246a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:00.360136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-acb1120c-c134-46dc-af5a-7360420246a3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:00.361349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-acb1120c-c134-46dc-af5a-7360420246a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:00.438826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:00.468489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 564/1110] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:00.620287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:00.621827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:00.639595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:00.769954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b9581-6dad-4d3e-9d18-20f7009a1201 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:00.859786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:00.887085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:00.887787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:00.888026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:00.888354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:00.890037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:00.985466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:00.987034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a0fc-402d-4d69-8d71-46169a7b4fb7 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:01.012775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 547/1111] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:21:02.058565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:02.059424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:02.059424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:02.061614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 565/1112] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:21:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:02.071905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d0c3-118e-49df-8c47-a955aaaee382 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:02.072488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d0c3-118e-49df-8c47-a955aaaee382 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:02.072488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d0c3-118e-49df-8c47-a955aaaee382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:02.159187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-42a9d0c3-118e-49df-8c47-a955aaaee382 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:02.160469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-42a9d0c3-118e-49df-8c47-a955aaaee382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:02.206590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:02.264865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 548/1113] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:02.416016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:02.417476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:02.573658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59dd52-6476-430a-a9a5-27d7639c9831 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:02.629245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:02.689717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:02.690345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:02.690629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:02.690890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:02.692526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:02.799534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:02.801207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abada4b2-632e-4a76-a807-0badee8738ae service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:02.820244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 566/1114] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:03.873699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:03.874552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:03.874552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:03.876730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 549/1115] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:21:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:03.883087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9120-9fd5-4702-a3db-13a6660fc3d0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:03.883552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9120-9fd5-4702-a3db-13a6660fc3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:03.883657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a9120-9fd5-4702-a3db-13a6660fc3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:03.967393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-701a9120-9fd5-4702-a3db-13a6660fc3d0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:03.968626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-701a9120-9fd5-4702-a3db-13a6660fc3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:03.970577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:04.066336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 567/1116] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:04.217848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:04.220323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:04.379627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5a045-a18d-478b-b60e-6eec3949ca69 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:04.410885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:04.501386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:04.501908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:04.502189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:04.502455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:04.504066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:04.597607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:04.599193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffea6d-895f-4fbd-8b28-bb1e4b45097f service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:04.617617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 550/1117] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:06.660964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:06.661583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:06.661583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:06.663496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 568/1118] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:06.669140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242fb3-2da0-43ba-ba14-6dfe4281e1b3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:06.669588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242fb3-2da0-43ba-ba14-6dfe4281e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:06.669697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242fb3-2da0-43ba-ba14-6dfe4281e1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:06.714769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:06.742563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-41242fb3-2da0-43ba-ba14-6dfe4281e1b3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:06.743784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-41242fb3-2da0-43ba-ba14-6dfe4281e1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:06.842039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 551/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:07.123175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:07.125698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:07.238183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ceaf9-1cd7-490b-867d-8dc428c81da1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:07.275360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:07.323547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:07.324213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:07.324508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:07.324782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:07.326392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:07.450612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:07.452057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6accc-38f0-484e-82d5-9ae636ab5f9c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:07.473692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 569/1120] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:09.749622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:09.750436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:09.750436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:09.752872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 552/1121] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:09.761883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddef6e-14d2-450b-8f8f-604dfe53d5b9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:09.762317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddef6e-14d2-450b-8f8f-604dfe53d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:09.762481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddef6e-14d2-450b-8f8f-604dfe53d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:09.833552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0fddef6e-14d2-450b-8f8f-604dfe53d5b9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:09.834762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0fddef6e-14d2-450b-8f8f-604dfe53d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:09.848837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:09.945696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 570/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:10.228503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:10.230064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:10.342375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c89e1-7c56-4a81-9d85-32daa6756193 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:10.355810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:10.424133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:10.424789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:10.425065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:10.425331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:10.426986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:10.542572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:10.543927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba910d2-0d08-4e82-bdff-2d0cd9fc6af9 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:10.561960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 553/1123] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:13.032653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c13f-2050-421e-9340-aa7923096279 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:13.033229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c13f-2050-421e-9340-aa7923096279 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:13.033229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c13f-2050-421e-9340-aa7923096279 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:13.034778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 571/1124] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:13.040963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a947-6202-44b0-b5f8-820e770bdec7 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:13.041401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a947-6202-44b0-b5f8-820e770bdec7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:13.041538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a947-6202-44b0-b5f8-820e770bdec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:13.101784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:13.121179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8523a947-6202-44b0-b5f8-820e770bdec7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:13.122413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8523a947-6202-44b0-b5f8-820e770bdec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:13.227468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 554/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:13.524807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c13f-2050-421e-9340-aa7923096279 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:13.526523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f612c13f-2050-421e-9340-aa7923096279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:13.635305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c13f-2050-421e-9340-aa7923096279 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:13.678518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:13.718200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:13.718812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:13.719089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:13.719353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:13.720977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:13.837376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:13.838754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17318c-c3ad-4c14-bfad-54daa3dfd8a7 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:13.856778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 572/1126] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:16.146721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:16.147317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:16.147317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:16.148651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 555/1127] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:16.155521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f966-23bd-46cf-97ee-7c83a0634477 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:16.156241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f966-23bd-46cf-97ee-7c83a0634477 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:16.156429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f966-23bd-46cf-97ee-7c83a0634477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:16.228341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-54d5f966-23bd-46cf-97ee-7c83a0634477 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:16.229543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-54d5f966-23bd-46cf-97ee-7c83a0634477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:16.250388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:16.333674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 573/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:16.617894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:16.619598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:16.732817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e91021-7fed-4bc6-8334-ad898d308dcc None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:16.753884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:16.808435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:16.809049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:16.809325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:16.809593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:16.811205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:16.926244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:16.927659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0630a-45a0-4ee7-960d-95984f6a1a98 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:16.947494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 556/1129] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:19.173715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:19.174287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:19.174287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:19.175610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 574/1130] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:19.182166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da49c-c731-45c9-874a-304e3d422c90 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:19.182882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da49c-c731-45c9-874a-304e3d422c90 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:19.183160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991da49c-c731-45c9-874a-304e3d422c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:19.252908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:19.255591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-991da49c-c731-45c9-874a-304e3d422c90 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:19.257380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-991da49c-c731-45c9-874a-304e3d422c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:19.367052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 557/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:19.652465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:19.654687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:19.774006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08593c-f6bb-4fc3-91ef-7871ed2e1c6e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:19.826537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:19.860181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:19.860733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:19.861000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:19.861266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:19.862869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:19.975328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:19.976830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfd090-ade9-4774-ae85-d9ba40492401 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:19.995822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 575/1132] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:22.192196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:22.193113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:22.193113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:22.195299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 558/1133] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:22.200960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541b88-c8c2-4ca8-b2ce-781908fd1fd1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:22.201437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541b88-c8c2-4ca8-b2ce-781908fd1fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:22.201553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541b88-c8c2-4ca8-b2ce-781908fd1fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:22.270985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-58541b88-c8c2-4ca8-b2ce-781908fd1fd1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:22.272216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58541b88-c8c2-4ca8-b2ce-781908fd1fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:22.309226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:22.380492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 576/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:22.671899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:22.673533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:22.783246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554df98-2eb1-4328-8ad3-118213ecb7a5 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:22.813854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:22.860989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:22.861524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:22.861798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:22.862069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:22.863737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:22.981911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:22.983326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac25537-af6f-40d9-8bd5-a2b7bf79c255 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:23.002185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 559/1135] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:21:25.163107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:25.163619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:25.163619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:25.165558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 577/1136] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:25.171425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4066f6-551e-4b34-91a1-1f2187f1ba39 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:25.171906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4066f6-551e-4b34-91a1-1f2187f1ba39 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:25.172011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4066f6-551e-4b34-91a1-1f2187f1ba39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:25.237666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5f4066f6-551e-4b34-91a1-1f2187f1ba39 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:25.238848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5f4066f6-551e-4b34-91a1-1f2187f1ba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:25.247234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:25.339546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 560/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:25.622168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:25.623816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:25.730289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a313-d645-4cc2-8b66-7f9883fbcbf5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:25.788422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:25.806304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:25.806909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:25.807181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:25.807446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:25.809067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:25.921814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:25.923156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e66e80-a828-4d58-9d0c-51f7ccc5394f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:25.941557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 578/1138] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:28.232812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:28.233409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:28.233409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:28.234728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 561/1139] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:28.240617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708769-b070-4de0-bce8-6d1f787fbe4e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:28.241062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708769-b070-4de0-bce8-6d1f787fbe4e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:28.241179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708769-b070-4de0-bce8-6d1f787fbe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:28.315105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-39708769-b070-4de0-bce8-6d1f787fbe4e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:28.316349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-39708769-b070-4de0-bce8-6d1f787fbe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:28.355438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:28.414944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 579/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:28.704250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:28.705830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:28.812360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ec345-20fc-4256-a3ae-0de40bb833e6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:28.851533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:28.887230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:28.887845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:28.888099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:28.888396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:28.890239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:29.009950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:29.011378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af7717-cd30-408e-a546-4470353f2e2e service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:29.030185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 562/1141] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:31.181962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:31.182579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:31.182579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:31.183853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 580/1142] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:31.189592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3608b6-3cb0-421d-a7d9-b19b9cac43d6 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:31.190037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3608b6-3cb0-421d-a7d9-b19b9cac43d6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:31.190139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3608b6-3cb0-421d-a7d9-b19b9cac43d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:31.257921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5f3608b6-3cb0-421d-a7d9-b19b9cac43d6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:31.259187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5f3608b6-3cb0-421d-a7d9-b19b9cac43d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:31.277961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:31.359254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 563/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:31.651908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:31.653493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:31.776344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c5bb8-8093-4cd6-bf00-e08ffffe3474 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:31.848106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:31.856023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:31.856670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:31.856972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:31.857240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:31.858898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:31.984773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:31.986226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d165c-1ef8-4b78-abf0-a6d1ea2b9fa6 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:32.005226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 581/1144] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:34.141768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:34.142982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:34.142982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:34.145560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 564/1145] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:21:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:34.151980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9722-2c19-40d8-a084-389abd324f73 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:34.152728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9722-2c19-40d8-a084-389abd324f73 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:34.152898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9722-2c19-40d8-a084-389abd324f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:34.223624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b12e9722-2c19-40d8-a084-389abd324f73 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:34.224842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b12e9722-2c19-40d8-a084-389abd324f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:34.281222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:34.336459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 582/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:34.618872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:34.620473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:34.737375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd30a2-9e0b-49c9-a458-64ea3a24c61f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:34.790038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:34.817540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:34.818174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:34.818446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:34.818714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:34.820335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:34.942401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:34.943820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f36a37-00ee-444c-bff5-0b375ad2bad8 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:34.963476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 565/1147] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:44.334657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=22 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:44.960537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=66 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:54.796814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6214c31-239e-4300-84d6-6cb2aefd8898 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:54.797436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6214c31-239e-4300-84d6-6cb2aefd8898 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:54.797436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6214c31-239e-4300-84d6-6cb2aefd8898 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:54.798703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 583/1148] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:54.807115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bbfa-f377-4613-acd7-9143d92f38f4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:54.807559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bbfa-f377-4613-acd7-9143d92f38f4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:54.807670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bbfa-f377-4613-acd7-9143d92f38f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:54.878436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0d39bbfa-f377-4613-acd7-9143d92f38f4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:54.879582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0d39bbfa-f377-4613-acd7-9143d92f38f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:54.983050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 566/1149] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:55.287644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:55.287922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:55.288214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:55.289560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 584/1150] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 11 13:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:55.294828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f88aa-d551-45e3-98a6-98046a7c838e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:55.295107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f88aa-d551-45e3-98a6-98046a7c838e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:55.295379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f88aa-d551-45e3-98a6-98046a7c838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:55.372022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f80f88aa-d551-45e3-98a6-98046a7c838e None None] MFA Rules not processed for user `f056f1106e9a4b5ba29fe99058e47c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:55.373256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f80f88aa-d551-45e3-98a6-98046a7c838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:55.395358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:55.454125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 567/1151] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 11 13:21:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:55.462101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:55.463608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:55.571473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58167cd4-63aa-41d0-a01d-52031b018285 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:55.651272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:55.651880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:55.652151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:55.652453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:55.654046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:55.709471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:55.767154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:55.768954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef5419-7080-4cab-9e15-6276554c944f service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:55.787653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 585/1152] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:21:58.026495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:58.027430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:58.027430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:58.029611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 568/1153] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:21:58.036095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d366-fccd-4a2b-b960-5d08fd54faed None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:58.036584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d366-fccd-4a2b-b960-5d08fd54faed None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:58.036829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d366-fccd-4a2b-b960-5d08fd54faed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:58.107946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6ee7d366-fccd-4a2b-b960-5d08fd54faed None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:21:58.109176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6ee7d366-fccd-4a2b-b960-5d08fd54faed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:58.212358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 586/1154] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:21:58.494111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:21:58.496609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:21:58.637999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b4ed2-9082-4156-9909-d87c0131a24a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:21:58.694679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:21:58.745100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:21:58.745623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:21:58.745894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:21:58.746144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:21:58.747770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:21:58.748076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:21:58.748076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:21:58.748076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:21:58.749255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2c853-4378-49e3-816e-94034d295e10 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:21:58.769480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 569/1155] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 11 13:21:58 2022] GET /identity/v3/projects => generated 2553 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:22:00.942083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:00.942651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:00.942651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:00.943944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 587/1156] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:22:00.950513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ea10d-88c1-4471-81e0-348c741d1f8a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:00.950962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ea10d-88c1-4471-81e0-348c741d1f8a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:00.951062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ea10d-88c1-4471-81e0-348c741d1f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:01.029512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7c2ea10d-88c1-4471-81e0-348c741d1f8a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:22:01.030708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7c2ea10d-88c1-4471-81e0-348c741d1f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:01.135216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 570/1157] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:22:01.436715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:22:01.438304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:01.448504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:01.596922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8201e7-90fb-442a-a15d-acaf574d3b61 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:22:01.687366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:01.731348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:22:01.731966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:01.732236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:01.732534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:01.734112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:01.825789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:22:01.827368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8383f-cc1a-44cc-a828-72786c87019c service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:22:01.845319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 588/1158] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:22:04.133998 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:04.134600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:04.134600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:04.135877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 571/1159] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:22:04.142377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bd98-39f1-45a5-bd18-6fbe33776855 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:04.143126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bd98-39f1-45a5-bd18-6fbe33776855 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:04.143226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3bd98-39f1-45a5-bd18-6fbe33776855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:04.219771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-12a3bd98-39f1-45a5-bd18-6fbe33776855 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:22:04.220996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-12a3bd98-39f1-45a5-bd18-6fbe33776855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:04.271457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:04.322823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 589/1160] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:22:04.608682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:22:04.610247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:04.634624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:04.721609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97c331-aac2-4e71-9f34-1b04daff0694 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:22:04.801088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:22:04.801698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:04.801975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:04.802244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:04.803873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:04.878343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:04.914911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:22:04.916713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580ae0d-241f-43c5-bee7-81bd6640b143 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:22:04.934804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 572/1161] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:22:07.126532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:07.127442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:07.127442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:07.129781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 590/1162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:22:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:22:07.135544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d78ae9-bdd2-4bdc-9048-b34b60c5a432 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:07.135997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d78ae9-bdd2-4bdc-9048-b34b60c5a432 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:07.136107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d78ae9-bdd2-4bdc-9048-b34b60c5a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:07.206467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f8d78ae9-bdd2-4bdc-9048-b34b60c5a432 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:22:07.207678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f8d78ae9-bdd2-4bdc-9048-b34b60c5a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:07.311028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 573/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:22:07.591595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:22:07.593164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:07.709568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6364c3-7f46-4ba8-9070-20e285a0fc43 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:22:07.790564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:07.792099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:22:07.792769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:07.793068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:07.793364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:07.795559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:07.905864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:22:07.907230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80089-5f75-4173-8c00-a8f2a44671fc service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:22:07.925839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 591/1164] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:22:10.229919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:10.230819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:10.230819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:10.233099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 574/1165] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:22:10.238670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461962-53bd-4663-802e-192c398a90a0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:10.239109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461962-53bd-4663-802e-192c398a90a0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:10.239225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461962-53bd-4663-802e-192c398a90a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:10.309705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9e461962-53bd-4663-802e-192c398a90a0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:22:10.310908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9e461962-53bd-4663-802e-192c398a90a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:10.364197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:10.408710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 592/1166] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:22:10.697308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:22:10.698861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:10.737583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:10.807892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fb1a1-0442-4628-a172-a04a5327c6b3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:22:10.881894 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:22:10.882495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:10.882764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:10.883049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:10.884708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:10.972970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:11.003578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:22:11.005014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644e9ce-bb01-42f9-a48c-cae4d63880e0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:22:11.025305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 575/1167] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:22:13.399807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:13.400398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:13.400398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:13.401646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 593/1168] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 11 13:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:22:13.408306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b70537-5799-430b-b72e-aacc0be8da3c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:13.409003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b70537-5799-430b-b72e-aacc0be8da3c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:13.409200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b70537-5799-430b-b72e-aacc0be8da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:13.487873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-52b70537-5799-430b-b72e-aacc0be8da3c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:22:13.489138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-52b70537-5799-430b-b72e-aacc0be8da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:13.590019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 576/1169] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 11 13:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:22:13.887966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:22:13.890355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:14.005623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d54957-83db-4101-9431-b0ea120bbc46 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:22:14.082460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:22:14.083073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:22:14.083372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:22:14.083636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:22:14.085270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:22:14.097462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:14.204565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:22:14.206007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bde3f-d411-4c7c-b4c7-583f54544adf service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:22:14.224941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 594/1170] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:22:23.587930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=82 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:22:24.222554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=50 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:36.402732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:36.403737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:36.403737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:36.485573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:23:36.486775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:36.592343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 577/1171] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:23:36.600579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:36.601103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:36.601103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:36.686698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:23:36.687911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:36.791968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 595/1172] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:23:36.797659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:36.799545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:36.813792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76dd56-d118-4920-8ce0-9ffbdef74c1f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:36.879655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:36.919539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:36.920161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:36.920464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:36.920739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:36.921394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:23:36.921700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:36.922721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:23:36.922721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:23:36.922721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:23:36.923623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:36.935078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 578/1173] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:23:36.944862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:36.954252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb851fa-cfc4-4480-ad83-f5b07f641203 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:37.018691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:37.052871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:37.053487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.053763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.054027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.054663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:37.055810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:23:37.055810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:23:37.055810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:23:37.057040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:37.127347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 596/1174] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:23:36 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:23:37.133674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:37.135318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:37.171718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0007e-44cc-426a-8f90-ce61d27ae64c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:37.271614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:37.272155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.272450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.272722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.273351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:37.274439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:23:37.274439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:23:37.274439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:23:37.275643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:37.322374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 579/1175] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:23:37 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:23:37.327838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:37.329442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:37.400917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2244e1d-94cc-475e-97c2-d5d6ae7d374a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:37.486626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:37.498241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:37.498851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.499127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.499397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.500034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:37.501677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:37.514318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 597/1176] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:23:37.518868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:37.520430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:37.532107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe3beb-ef0d-43f1-bc26-79cc322c90b9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:37.557992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:37.634392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:37.635006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.635281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.635544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.636218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:37.637826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8143d8-830b-4177-8454-aaf5412a3165 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:37.650588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 580/1177] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:23:37.655042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:37.656595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:37.691947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a623c4-b7cf-4fa6-9703-1a6fbb72c08e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:37.787148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:37.787748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.788011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.788301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] PATH_INFO: `/v3/projects/2daa1c034e8e460fb18628d550b801c8/users/f111c11ecc16432c9646103fa06acbef/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.811173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2daa1c034e8e460fb18628d550b801c8, user_id=f111c11ecc16432c9646103fa06acbef, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:37.813848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c7423-c4b6-4b3f-af38-f7afdf5eaa57 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:37.841560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 598/1178] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:23:37 2022] PUT /identity/v3/projects/2daa1c034e8e460fb18628d550b801c8/users/f111c11ecc16432c9646103fa06acbef/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:23:37.853436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362da3f-bfe6-42b0-b1a7-f531d8f68a0d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:37.853725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362da3f-bfe6-42b0-b1a7-f531d8f68a0d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:37.853995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362da3f-bfe6-42b0-b1a7-f531d8f68a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:37.913499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c362da3f-bfe6-42b0-b1a7-f531d8f68a0d None None] MFA Rules not processed for user `f111c11ecc16432c9646103fa06acbef`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:23:37.914693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c362da3f-bfe6-42b0-b1a7-f531d8f68a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:37.993154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 581/1179] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:23:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:23:38.010785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.011062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.011329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.012705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 599/1180] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 11 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:23:38.017707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d19a5-63e8-49ad-be79-c77819b0d075 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.017990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d19a5-63e8-49ad-be79-c77819b0d075 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.018593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d19a5-63e8-49ad-be79-c77819b0d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.051175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:38.094828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-547d19a5-63e8-49ad-be79-c77819b0d075 None None] MFA Rules not processed for user `3ecc3d37e6214164a01eb96629509215`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:23:38.096014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-547d19a5-63e8-49ad-be79-c77819b0d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:38.178394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 582/1181] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 11 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:23:38.186327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:38.187811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:38.239791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:38.302610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db36cf-49c8-431f-8f61-9df6dbccf077 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:38.378477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:38.379076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.379345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.379608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.381216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:38.480941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:38.482395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:38.485789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:38.500252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 600/1182] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:23:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:23:38.547236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8107f-085c-4468-88c0-c1b633debce6 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.547624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8107f-085c-4468-88c0-c1b633debce6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.547783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8107f-085c-4468-88c0-c1b633debce6 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.549093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 583/1183] 10.222.0.15 () {52 vars in 830 bytes} [Tue Oct 11 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:23:38.552896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:38.554035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:38.657857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68bcd0-3689-4938-85df-1b95363eb590 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:38.760923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:38.761585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.761856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.762124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] PATH_INFO: `/v3/users/f111c11ecc16432c9646103fa06acbef` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.776597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f111c11ecc16432c9646103fa06acbef)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:38.778119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:23:38.778119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:23:38.778119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:23:38.779719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:38.791657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:38.852336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f111c11ecc16432c9646103fa06acbef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:38.866993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f111c11ecc16432c9646103fa06acbef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:38.892248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f111c11ecc16432c9646103fa06acbef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:38.892792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f111c11ecc16432c9646103fa06acbef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:38.903664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f111c11ecc16432c9646103fa06acbef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:38.909853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 601/1184] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:23:38 2022] DELETE /identity/v3/users/f111c11ecc16432c9646103fa06acbef => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:23:38.915399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:38.915726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:38.916082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:38.980002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:23:38.981237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:39.061352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:39.074470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 584/1185] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:23:39.084547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:39.085768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:39.190667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60971f-9009-4429-95e1-6cd1c9bbec7a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:39.255087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:39.278387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:39.279038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:39.279311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:39.279584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:39.281232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:39.397608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:39.399660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9cf4e-5b1f-4286-b63b-1b614fd23501 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:39.422407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 602/1186] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:23:39.781197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:23:39.782535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:23:39.796585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df605c-48bd-423b-8d48-735014cd213f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:23:39.897510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:23:39.898163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:23:39.898436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:23:39.898701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] PATH_INFO: `/v3/projects/2daa1c034e8e460fb18628d550b801c8` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:23:39.908827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2daa1c034e8e460fb18628d550b801c8)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:23:39.909835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:23:39.909835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:23:39.909835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:23:39.911026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:23:39.971383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2daa1c034e8e460fb18628d550b801c8', 'request_id': 'req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:39.984631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2daa1c034e8e460fb18628d550b801c8', 'request_id': 'req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:39.995012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2daa1c034e8e460fb18628d550b801c8', 'request_id': 'req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:40.023162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] The token cache is being invalidate because project 2daa1c034e8e460fb18628d550b801c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:23:40.023325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:40.023884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-376db7f1-9e72-4ef1-85bf-82e5e72d1f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:23:40.025234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 585/1187] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:23:39 2022] DELETE /identity/v3/projects/2daa1c034e8e460fb18628d550b801c8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:23:49.419841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=62 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:23:50.021016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:25:07.135256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:07.135897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:07.135897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:07.137174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 603/1188] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:07.142524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008beba-869b-4586-a7a1-8b6d2f4e8e8b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:07.143022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008beba-869b-4586-a7a1-8b6d2f4e8e8b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:07.143022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008beba-869b-4586-a7a1-8b6d2f4e8e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:07.228856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2008beba-869b-4586-a7a1-8b6d2f4e8e8b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:07.230069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2008beba-869b-4586-a7a1-8b6d2f4e8e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:07.349886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 586/1189] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:25:07.504245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:07.507924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:07.637101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0d2c-a6b5-4be4-959e-a4e8f758d518 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:07.644103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:07.644745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:07.645025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:07.645297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:07.653717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:07.655372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380428a-c82a-47e1-aa44-098ac31a00b8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:07.665346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 604/1190] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:08.734386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aeb92-5be2-46f6-8ebf-dbb8456eb528 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:08.735008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aeb92-5be2-46f6-8ebf-dbb8456eb528 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:08.735008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aeb92-5be2-46f6-8ebf-dbb8456eb528 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:08.736418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 587/1191] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:08.743085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0b02d-1220-4c1b-ae11-aef2536787aa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:08.743580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0b02d-1220-4c1b-ae11-aef2536787aa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:08.743580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0b02d-1220-4c1b-ae11-aef2536787aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:08.763931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ec0b02d-1220-4c1b-ae11-aef2536787aa None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:08.765206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ec0b02d-1220-4c1b-ae11-aef2536787aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:08.773620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 605/1192] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:08.925973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aeb92-5be2-46f6-8ebf-dbb8456eb528 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:08.946287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aeb92-5be2-46f6-8ebf-dbb8456eb528 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:08.953027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:08.953637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:08.953910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:08.954178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:08.963038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:08.964651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5287c1-1ed2-40f9-8fe1-966b52650b55 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:08.974173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 588/1193] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:09.322428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b26d-c511-4b23-9e83-951e9e19284f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:09.322639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b26d-c511-4b23-9e83-951e9e19284f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:09.322852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109b26d-c511-4b23-9e83-951e9e19284f None None] PATH_INFO: `/v3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:09.324654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 606/1194] 10.222.0.36 () {54 vars in 925 bytes} [Tue Oct 11 13:25:09 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:25:09.331316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc026-da92-4d48-bd34-fe56431d161e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:09.331600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc026-da92-4d48-bd34-fe56431d161e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:09.331871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc026-da92-4d48-bd34-fe56431d161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:09.393808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-055dc026-da92-4d48-bd34-fe56431d161e None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:09.395000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-055dc026-da92-4d48-bd34-fe56431d161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:09.412600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:25:09.471464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 589/1195] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 11 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:13.042544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911de9-6d5f-439d-8259-25fbf538e5e9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:13.043099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911de9-6d5f-439d-8259-25fbf538e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:13.043099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911de9-6d5f-439d-8259-25fbf538e5e9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:13.044662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 607/1196] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:13.051302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f56c-6645-4abe-9cdd-b973a84fbb7b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:13.051831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f56c-6645-4abe-9cdd-b973a84fbb7b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:13.052071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f56c-6645-4abe-9cdd-b973a84fbb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:13.074012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-86a9f56c-6645-4abe-9cdd-b973a84fbb7b None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:13.075229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-86a9f56c-6645-4abe-9cdd-b973a84fbb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:13.084720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 590/1197] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:13.092518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911de9-6d5f-439d-8259-25fbf538e5e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:13.108095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911de9-6d5f-439d-8259-25fbf538e5e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:13.114000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:13.114599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:13.114872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:13.115133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:13.123687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:13.125329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47747ab0-ea71-4f5f-b17a-a922c5b8edb4 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:13.134666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 608/1198] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:20.444690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1725f1-19ee-4c00-9dc9-01c0deb8e571 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:20.445292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1725f1-19ee-4c00-9dc9-01c0deb8e571 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:20.445292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1725f1-19ee-4c00-9dc9-01c0deb8e571 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:20.446595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 591/1199] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:20.453955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55548ef-dd57-4932-a286-9dab890a2aec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:20.454803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55548ef-dd57-4932-a286-9dab890a2aec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:20.454803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55548ef-dd57-4932-a286-9dab890a2aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:20.477475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b55548ef-dd57-4932-a286-9dab890a2aec None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:20.478680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b55548ef-dd57-4932-a286-9dab890a2aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:20.488304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 609/1200] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:20.643908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1725f1-19ee-4c00-9dc9-01c0deb8e571 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:20.663876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1725f1-19ee-4c00-9dc9-01c0deb8e571 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:20.670071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:20.670673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:20.670945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:20.671212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:20.679375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:20.680985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652643f-343c-4031-be9e-e79281ef451e service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:20.690992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 592/1201] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:21.753141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27b28-e979-450b-be00-08ac19016b56 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:21.754056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27b28-e979-450b-be00-08ac19016b56 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:21.754056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27b28-e979-450b-be00-08ac19016b56 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:21.756267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 610/1202] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 13:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:21.763478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11949127-7194-4bbd-945a-437e2597809d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:21.763988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11949127-7194-4bbd-945a-437e2597809d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:21.764094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11949127-7194-4bbd-945a-437e2597809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:21.786258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-11949127-7194-4bbd-945a-437e2597809d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:21.787474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-11949127-7194-4bbd-945a-437e2597809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:21.796420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 593/1203] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:21.951355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa27b28-e979-450b-be00-08ac19016b56 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:21.968775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa27b28-e979-450b-be00-08ac19016b56 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:21.975263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:21.975862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:21.976147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:21.976436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:21.984479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:21.986089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705047d-b8a1-43de-972c-ec11a539feb9 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:21.995575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 611/1204] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:26.063075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:26.063661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:26.063661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:26.065083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 594/1205] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:26.070497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e38faf-98c6-4c66-abe2-a1d008c3c700 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:26.070974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e38faf-98c6-4c66-abe2-a1d008c3c700 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:26.071082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e38faf-98c6-4c66-abe2-a1d008c3c700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:26.091303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d7e38faf-98c6-4c66-abe2-a1d008c3c700 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:26.092428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d7e38faf-98c6-4c66-abe2-a1d008c3c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:26.100978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 612/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:26.389618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:26.392782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:26.476588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844509b8-a174-44f0-b7a2-18ce5034d690 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:26.480313 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:26.480923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:26.481192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:26.481459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:26.490938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:26.492374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e80401-b218-499f-8405-8b0182f7ea49 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:26.501463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 595/1207] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:28.606526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:28.607001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:28.607001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:28.608446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 613/1208] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:28.614651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757dd7-6a09-4236-97e1-11f8ddde012c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:28.615075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757dd7-6a09-4236-97e1-11f8ddde012c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:28.615243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757dd7-6a09-4236-97e1-11f8ddde012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:28.645388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b9757dd7-6a09-4236-97e1-11f8ddde012c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:28.646596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b9757dd7-6a09-4236-97e1-11f8ddde012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:28.656029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 596/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:28.940096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:28.942350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:28.958158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071799d-8c27-459f-96fc-2dac0e97a9f0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:28.962513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:28.963117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:28.963394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:28.963664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:28.973582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:28.975015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b525fff-92b2-4d0e-afe5-148eb72bb74a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:28.984769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 614/1210] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:32.647027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:32.647912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:32.647912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:32.650162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 597/1211] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:32.658252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddfd9-c410-46a9-a8b1-6547f82180f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:32.658679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddfd9-c410-46a9-a8b1-6547f82180f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:32.658813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddfd9-c410-46a9-a8b1-6547f82180f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:32.666761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:25:32.679995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e7bddfd9-c410-46a9-a8b1-6547f82180f1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:32.681238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e7bddfd9-c410-46a9-a8b1-6547f82180f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:32.690422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 615/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:25:32.991134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:32.993460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:33.062580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:25:33.076331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c24f1-9870-41bb-b177-b35801a77a7a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:33.079943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:33.080552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:33.080831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:33.081139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:33.090048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:33.091463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf858b-f69a-47a3-a8b2-9d7eff9e171a service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:33.101107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 598/1213] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:35.231541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:35.232177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:35.232177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:35.233463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 616/1214] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 13:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:25:35.239702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432721-6482-4795-923c-de1b8b6eb95f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:35.240365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432721-6482-4795-923c-de1b8b6eb95f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:35.240476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432721-6482-4795-923c-de1b8b6eb95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:35.267399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-79432721-6482-4795-923c-de1b8b6eb95f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:25:35.268972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-79432721-6482-4795-923c-de1b8b6eb95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:35.277283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 599/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:25:35.576753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:25:35.578816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:25:35.599478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e3cf5-a11a-4fd8-beda-7ab6ad4ae7db None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:25:35.603820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:25:35.604467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:25:35.604755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:25:35.605018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:25:35.614578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:25:35.615998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e013-09b6-47af-b0c7-3a9d1ba4dfa4 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:25:35.625753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 617/1216] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:25:45.261966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=11 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:25:45.621533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:58.475970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0b8d-70a8-49cc-bfdc-61588dec7f48 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.479041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0b8d-70a8-49cc-bfdc-61588dec7f48 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.479041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e0b8d-70a8-49cc-bfdc-61588dec7f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.504976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-388e0b8d-70a8-49cc-bfdc-61588dec7f48 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.506552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-388e0b8d-70a8-49cc-bfdc-61588dec7f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.520715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 600/1217] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.531494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.532638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.533147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.546771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d014-5d5d-4b24-8bc1-9ada47be6de8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.548249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d014-5d5d-4b24-8bc1-9ada47be6de8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.548791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0d014-5d5d-4b24-8bc1-9ada47be6de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.565505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.573464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.582310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6bf0d014-5d5d-4b24-8bc1-9ada47be6de8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.583544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6bf0d014-5d5d-4b24-8bc1-9ada47be6de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.591362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 618/1218] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.599968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 601/1219] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.601476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:58.611646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.611929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.612428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.614890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482dffd-1380-42a8-aacb-4f2536c67ed1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:58.624368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:58.625224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.625634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.626002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.626766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:58.627168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:58.628319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:26:58.628319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:26:58.628319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:26:58.629451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09e930-a44a-46d7-9d00-44bd20e87e7e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:58.644761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 619/1220] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:58.649342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.650563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.652819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.653356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.653858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.670249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 602/1221] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 11 13:26:58.671102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:58.684032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4687e-1a7c-4b0c-8aa2-fcab8f8f7d5d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:58.687638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.689212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.731916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:58.732656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.732942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.733243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.733990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:58.737173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 620/1222] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 11 13:26:58.738117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:58.739304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:26:58.739304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:26:58.739304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:26:58.743951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc711f6-2ecf-4d12-ac9e-0bf155450de0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:58.764157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f0b07-d0a3-4855-8417-61869b95c6d1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:58.772949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:58.773791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.773916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.774194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.774867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:58.776214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:58.777822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324b01d-b702-4334-b016-457b564641f5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:58.798014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 621/1223] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:58.799597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eff55-fb15-4795-8f14-05a144424355 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.799976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eff55-fb15-4795-8f14-05a144424355 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.800295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eff55-fb15-4795-8f14-05a144424355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.810932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 603/1224] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:58.814185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ecf5-db7a-4b9d-a27a-dd90b97d8190 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.814185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ecf5-db7a-4b9d-a27a-dd90b97d8190 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.814185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ecf5-db7a-4b9d-a27a-dd90b97d8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.821318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-362eff55-fb15-4795-8f14-05a144424355 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.822522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-362eff55-fb15-4795-8f14-05a144424355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.831572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2ba2ecf5-db7a-4b9d-a27a-dd90b97d8190 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.832772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2ba2ecf5-db7a-4b9d-a27a-dd90b97d8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.882021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 622/1225] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.883431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1d5d-1165-4783-a09a-c0a25a2c0901 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.883814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1d5d-1165-4783-a09a-c0a25a2c0901 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.884327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1d5d-1165-4783-a09a-c0a25a2c0901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.893813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 604/1226] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.895254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589dbd-2df2-4b70-ac55-aa555c4de8c7 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.895546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589dbd-2df2-4b70-ac55-aa555c4de8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.895941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589dbd-2df2-4b70-ac55-aa555c4de8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.904900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7feb1d5d-1165-4783-a09a-c0a25a2c0901 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.906186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7feb1d5d-1165-4783-a09a-c0a25a2c0901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.914518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 623/1227] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.915550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.915831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.916012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-07589dbd-2df2-4b70-ac55-aa555c4de8c7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.916194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.917276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-07589dbd-2df2-4b70-ac55-aa555c4de8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.925627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 605/1228] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.926889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.927173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.927449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.937855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.939072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.946969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:58.947228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 624/1229] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.948058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:58.948273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:58.956706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 606/1230] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:58.957156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:58.958253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c06627-5dc2-4737-883d-3e957f38a607 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:58.963144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:58.963844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.964314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.964627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.965594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:58.966572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cf5b-d24a-4839-bff3-c6287ffd20ce None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:58.967508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458a444-082e-4866-8148-8a8ee25e36b5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:58.971289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:58.971966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.972298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.972556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:58.973192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:58.975075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d2445-10ee-4849-97aa-07dfa06d024c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:58.996271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 607/1231] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/users => generated 293 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:58.997224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 625/1232] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:58.997717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:58.997978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:58.998382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.001573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a6edc-17f0-474e-b33a-1c58032af21a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.001866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a6edc-17f0-474e-b33a-1c58032af21a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.002144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a6edc-17f0-474e-b33a-1c58032af21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.017276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.018786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.021349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3e3a6edc-17f0-474e-b33a-1c58032af21a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.022570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e3a6edc-17f0-474e-b33a-1c58032af21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.080349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 608/1233] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.081173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.085234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 626/1234] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.086475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32e49-a5b6-4a2e-9fee-9287e6fecb94 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.086755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32e49-a5b6-4a2e-9fee-9287e6fecb94 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.087022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32e49-a5b6-4a2e-9fee-9287e6fecb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.094323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc9648-b0a3-47cd-bbdb-1da965a8eb02 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.099284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.099915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.100195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.100472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.101189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.101472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.103085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f280b01-c4a0-4517-a30d-716e2c293567 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.107333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ccd32e49-a5b6-4a2e-9fee-9287e6fecb94 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.108586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ccd32e49-a5b6-4a2e-9fee-9287e6fecb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.115370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 609/1235] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.116473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.116756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.117031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.124679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 627/1236] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.129511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.129781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.130069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.130917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.137864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.139537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.147948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 610/1237] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.148834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.152599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.153771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.158371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f969f-2f25-4dd5-ad74-abc95fa3547a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.161550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 628/1238] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.162328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.167151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.167970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.168278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.168550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.169196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.170777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.175771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099eec6c-2f08-4609-bbd6-ea37a5befa08 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.180794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.181064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 611/1239] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:26:59.181510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.181787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.181880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.182097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.182766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.183825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:26:59.183825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:26:59.183825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:26:59.185059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.191426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6f293-97b8-46ba-baed-ad0faf28dffd admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.196220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.196950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.197152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.197384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.198060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.198297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.198429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.199907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.214218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 629/1240] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 282 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.214218 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.214508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 612/1241] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.214508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.222374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6b54e-b838-4f89-b386-60fc13ef07d9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.227752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.228403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.228683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.228951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.229658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ab0a-d850-46df-8252-fbf12802596b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.229778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.229974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.231591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.234308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.234904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.235182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.235451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.236085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.237973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.244788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 630/1242] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.245599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.255415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409ad7-08ac-4321-814c-6a827e403adc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.260359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.261094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.261401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.261762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.262502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.263189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.264943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.271234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 613/1243] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.272075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.278323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 631/1244] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.279162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.289982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7b069-dff1-441c-acac-940c76223581 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.333707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705eb15-ce76-44f6-9028-1d9ff413b626 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.338701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.339465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.339896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.340141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.340956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.341266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.342867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.346528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.347296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.347578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.347841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.348591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:26:59.348897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.350546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.354298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 614/1245] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.355162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.363047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 632/1246] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:26:59.363924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.366820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c333e6-7f4a-4772-87a4-0f1a51ebd4ab admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.371906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.372543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.372823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.373097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.373549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16cf93-ba8e-405c-82b8-7fa8bdb86fd7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.373738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.375319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.378160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.378744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.379018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.379286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.379916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.381570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.386573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 615/1247] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:26:59.387411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.397423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 633/1248] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:26:59.397423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.397686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bac7b-9ee0-4cc8-96da-19a58bd08773 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.402128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.402726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.403003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.403271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.403890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.405249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34d8f4-8ab2-4647-aa55-85f4d81f5d74 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.405796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.409763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.410342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.410618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.410893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.411507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.413407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.436365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 616/1249] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.436365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.444019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 634/1250] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.444019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.498812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc536e-552c-4a0f-9bb5-1e930c145698 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.500630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742343f6-271f-445b-be1f-1556bdc7ca46 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.504149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.504770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.505070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.505275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.505405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.505970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.506131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.506230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.506440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.507058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.507876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.508993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.530800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 617/1251] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.534555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.537486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.544642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 635/1252] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.545194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aec88-475d-42f9-8bed-036bc9e25200 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.545312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.562695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.604909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.605539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.605819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.606092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.606861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.608845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.609827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17f9a-61b0-415f-a16a-4890cdc6d30d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.614857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.615471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.615748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.616012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.616675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.618585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.638471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 618/1253] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.639397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.650045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 636/1254] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.651049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.705824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75fd15-ff92-461c-9ad0-fa52aa8c0600 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.710552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.711217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.711493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.711764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] PATH_INFO: `/v3/projects/659a7ca9a3684db4b6a29c780e7f8eb7/users/2fc3a4b9afd14409b3aef17f49aafd29/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.712423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5ba41-2b20-4262-9648-23c614f1e513 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.717075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.717680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.717962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.718230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.718870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.720649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.727220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=659a7ca9a3684db4b6a29c780e7f8eb7, user_id=2fc3a4b9afd14409b3aef17f49aafd29, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.729897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.730461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 637/1255] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:26:59.730461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.742846 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb49b-fa84-45c4-a358-0a2545123700 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.744205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 619/1256] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:26:59 2022] PUT /identity/v3/projects/659a7ca9a3684db4b6a29c780e7f8eb7/users/2fc3a4b9afd14409b3aef17f49aafd29/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:26:59.744991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.801510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.802038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.802315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.802582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.803210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.805175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.807574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7e95-cda7-4223-9448-278d132a2b8d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.814519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.815110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.815384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.815655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.816319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.818227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.831250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 638/1257] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.832522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.840476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 620/1258] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.841506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.846310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.847484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef457-f832-46e7-92a1-506df91b1c60 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.851805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daaabd-73a0-4634-996f-65e5fa8f4dda admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.852657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.853367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.853551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.853922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.854750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.856422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.857461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.858071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.858348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.858620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.859244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.861156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.867529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 639/1259] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:26:59.868157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.878999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691d447-bd11-41ff-a5ae-504a5585064a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.886442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 621/1260] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.886442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.888454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.889058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.889396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.889675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.890308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.892582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.894339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3428f-ac47-4977-b026-c82ade00cd15 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.899909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.900847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.901132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.901405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.902401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.905568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a5ccc-7c3e-4ebb-a479-eebab6aecb75 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.912478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:26:59.921703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 640/1261] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 294 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.922564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.932803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17687dfa-00cf-4761-8b98-915393e7c172 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.933282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 622/1262] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/users => generated 283 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:26:59.934653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.934939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.935211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.940094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:26:59.940924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:26:59.941208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:26:59.941483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] PATH_INFO: `/v3/projects/266ea3e1b3fa4c76ab08cb65dedbb9b4/users/e352807f0fed42a7adc6985a43028dab/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:26:59.954398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:26:59.955608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:26:59.956618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] RBAC: Authorizing `identity:create_grant(project_id=266ea3e1b3fa4c76ab08cb65dedbb9b4, user_id=e352807f0fed42a7adc6985a43028dab, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:26:59.959296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:26:59.963087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 623/1263] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:26:59.964204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:26:59.974153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc7b77-c30d-4e1e-bcf3-9ecedfef3e37 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:26:59.975696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 641/1264] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:26:59 2022] PUT /identity/v3/projects/266ea3e1b3fa4c76ab08cb65dedbb9b4/users/e352807f0fed42a7adc6985a43028dab/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:26:59.976622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.035977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.036628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.036900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ec52-3e06-40b7-bc0e-68a5a2fb344e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.037032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.037171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.037820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.039409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.041655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.042263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.042566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.042810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.043507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.045282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.049996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 624/1265] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.051267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.056496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 642/1266] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.057493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.061950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f41b0-20cf-4bdb-aaab-23ab5a01e8b1 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.067126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.067279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdbee-e1bd-4e31-bf7f-c38ec5d81c06 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.067757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.068035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.068337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.068970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.070552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.072296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.072934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.073219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.073489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.074126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.075714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.080865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 625/1267] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.081664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.089472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 643/1268] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.090290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.091234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76aaa-196e-401b-9504-2e2e83474eb4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.096535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.097143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.097418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.097688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.098318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.099881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.100749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb2e53-bb56-4adf-852e-8ca8692cd74e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.107116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.108194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.108432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.108700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.109334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.110858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 626/1269] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.111672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.112440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41f805-eed1-45cf-afe9-f782d7936cc3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.126210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 644/1270] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.129579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.129864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.130132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.158895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] MFA Rules not processed for user `e352807f0fed42a7adc6985a43028dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:00.160618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:00.171835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315e9ce-4fca-457a-8db6-da739ec09fc4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.175844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.176493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.176772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.177042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.187464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.188913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.198812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 627/1271] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:00.199623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.209078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445aa53-53d1-435a-85ed-3e3d43681f0e service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.214910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.215521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.215798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.216072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.216772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.218401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.229606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 628/1272] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.230309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.232134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 645/1273] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:00.236341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.240426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4d1-8436-4a52-909e-325c9f71df0a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.246807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.247430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.247710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.247984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.248665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.248948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c90bee-70c7-4bf8-9863-850d350dfe6c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.250271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.255084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.256248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.256647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.256921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] PATH_INFO: `/v3/projects/a86c05caf37e459fbbd42bb6f62a6f90/users/656bc16f0dfa45168d9176fe93394245/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.265492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 629/1274] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.265492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.276026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86c05caf37e459fbbd42bb6f62a6f90, user_id=656bc16f0dfa45168d9176fe93394245, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.277065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e023d-520d-49c9-8638-f6902695c30e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.279812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.282426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.283044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.283317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.283596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.284289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.285872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.297562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 630/1276] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.297797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 646/1276] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/a86c05caf37e459fbbd42bb6f62a6f90/users/656bc16f0dfa45168d9176fe93394245/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.298472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.299213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.307373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:00.311594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f28a-dd0a-48ee-af40-e490380e7c42 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.317858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:00.372905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8f9b-e038-42c5-8f34-7b096c74af32 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.381203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.381872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.382284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.382557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.383242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.384915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.391362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.392415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.393265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.393942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.394888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.399906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.400004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 647/1277] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.400004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.410566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338b88-d992-4339-bee2-12d0e5ea35ea admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.417799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 631/1278] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:00.417799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.420480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.421264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.421546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.421826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] PATH_INFO: `/v3/projects/2adf8ac988764dc29c6f0442b2398124/users/e4e126b005e149258013ce3b6cf15524/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.426485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497da238-1ba7-4a5d-8420-f801841af41f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.431519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.432349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.432630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.432897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.433970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.436151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.438412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adf8ac988764dc29c6f0442b2398124, user_id=e4e126b005e149258013ce3b6cf15524, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.441113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.459676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 648/1279] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/2adf8ac988764dc29c6f0442b2398124/users/e4e126b005e149258013ce3b6cf15524/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.459676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.476233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 632/1280] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:00.477227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.534803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347e7d9-f085-428c-aeb0-88b4bfacedde admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.548556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3319-7529-499b-b6e9-946bb0d6bb25 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.554609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.556097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.556600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.557148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] PATH_INFO: `/v3/projects/46f14bd1e0fd419383ac2bf2ab87e6d0/users/b50f86c6209f4a728cf3681a7d0a6765/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.579501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f14bd1e0fd419383ac2bf2ab87e6d0, user_id=b50f86c6209f4a728cf3681a7d0a6765, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.582287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd93151-3e4f-4630-b246-1cbf877853d8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.605762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 633/1281] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/46f14bd1e0fd419383ac2bf2ab87e6d0/users/b50f86c6209f4a728cf3681a7d0a6765/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.607251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.607657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.607995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.632831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.634817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.635518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.636024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] PATH_INFO: `/v3/projects/33ec6c4b270341c5a4ebce407bb5cca9/users/9a4a84c1ffc4466687122a6c67536d9b/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.646530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] MFA Rules not processed for user `656bc16f0dfa45168d9176fe93394245`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:00.647995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:00.660966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ec6c4b270341c5a4ebce407bb5cca9, user_id=9a4a84c1ffc4466687122a6c67536d9b, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.664191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.681118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 649/1282] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/33ec6c4b270341c5a4ebce407bb5cca9/users/9a4a84c1ffc4466687122a6c67536d9b/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.682315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.701296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:00.707936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:00.728857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 634/1283] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:00.729824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.755727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8cd08-fe80-42b8-b76d-463e80d39414 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.761352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.762022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.762714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.762714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] PATH_INFO: `/v3/projects/b225bb837bdd4e6390c7b24cb239fc1d/users/5734434bbd3348319c23c8055ec2f0ac/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.780551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b225bb837bdd4e6390c7b24cb239fc1d, user_id=5734434bbd3348319c23c8055ec2f0ac, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.783665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85de4e-74ba-491c-80ae-bcf68b7cdba6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.798848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 650/1284] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/b225bb837bdd4e6390c7b24cb239fc1d/users/5734434bbd3348319c23c8055ec2f0ac/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.800290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda703eb-f76f-41e6-94e6-ff161bdacb0c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.800672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda703eb-f76f-41e6-94e6-ff161bdacb0c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.801041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda703eb-f76f-41e6-94e6-ff161bdacb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.813621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c1968-f861-4fa3-97a3-1edf178038fa None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:00.834540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cda703eb-f76f-41e6-94e6-ff161bdacb0c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:00.836990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cda703eb-f76f-41e6-94e6-ff161bdacb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:00.879891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:00.880765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.881526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.881848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] PATH_INFO: `/v3/projects/33afb7274a9146cdbee0eeb2e6f48785/users/d439d54a4f2348d78ddb84691f2a4f63/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.901308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=33afb7274a9146cdbee0eeb2e6f48785, user_id=d439d54a4f2348d78ddb84691f2a4f63, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:00.904339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:00.925521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 635/1285] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:00 2022] PUT /identity/v3/projects/33afb7274a9146cdbee0eeb2e6f48785/users/d439d54a4f2348d78ddb84691f2a4f63/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:00.925521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:00.926296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 651/1286] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:00.927697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11427ea9-b544-4a8e-95cd-e21efe2a8fcb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:00.927989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11427ea9-b544-4a8e-95cd-e21efe2a8fcb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:00.928354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11427ea9-b544-4a8e-95cd-e21efe2a8fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:00.953054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-11427ea9-b544-4a8e-95cd-e21efe2a8fcb None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:00.954398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-11427ea9-b544-4a8e-95cd-e21efe2a8fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:01.020997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf88a-b23a-4f77-86e3-829768f7f12a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.026401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.027154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.027650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.028061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.028892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.030868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.032968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 652/1287] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:01.037592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdefb39-0304-4d24-bdce-e3545c56c6e3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.037938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdefb39-0304-4d24-bdce-e3545c56c6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.038575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdefb39-0304-4d24-bdce-e3545c56c6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.065304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1fdefb39-0304-4d24-bdce-e3545c56c6e3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:01.066843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fdefb39-0304-4d24-bdce-e3545c56c6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:01.078504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 636/1288] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:00 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:01.079679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.080981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 653/1289] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:01.082452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.082784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.083057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.093782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d31cb-cd8f-4c5a-a4fe-85ff45d34706 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.101102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.102018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.102519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.103003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.103920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.107509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339a272-3b18-4c9e-8ef6-55edc07dffa3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.114450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] MFA Rules not processed for user `5734434bbd3348319c23c8055ec2f0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:01.115724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:01.120624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:01.140345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:01.160773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 637/1290] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:01 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:01.162815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.163152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.163562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.186222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:01.187486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:01.207874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 654/1291] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:01.209264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.276805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 638/1292] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:01.276805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.283431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e47bf-2659-403d-9607-966ef31a4d96 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.347443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049d76-5c3e-4b98-b800-f362f92d25f3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.351530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.352335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.352841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.353216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.365763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.366634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.367650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.367650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.367865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.367971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.377935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.379389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.380140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 639/1293] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:01.391652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 655/1294] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:01.392611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.393390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.402587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e93b-d16f-4a6d-9de0-67a4b795a2ed service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.408289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.409235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.409707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.410247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.414838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7436d0-e4be-43d5-ba73-29b9ea575a8f service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.419893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.421526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.421526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.421526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.421835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.423230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.433039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.435446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 640/1295] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:01.436191 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.438114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.450672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0252-e963-4a01-855f-7cae30b5a9bd admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.455402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 656/1296] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:01.456316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.463420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.464540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.464817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.465089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.465729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.472743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.518875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 641/1297] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:01 2022] POST /identity/v3/users => generated 293 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:01.520867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.545404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734660-f658-4e2a-9a9f-1d9b059dc19f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.549516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679e548-c2d8-4be0-8901-c9301ae96afe service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.554108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.554901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.555042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.556413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.559455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.560616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.561165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.561967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.582532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.591047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.602965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 642/1298] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:01.603728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.614878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b1ef0-74b3-4d47-9e6f-cdfa926f6938 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.617176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:01.620838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.621535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.621807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.622107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] PATH_INFO: `/v3/projects/85f464516da748bca9958cd257c72e35/users/e2f2371c92474e9a905406950c35d170/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.645072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f464516da748bca9958cd257c72e35, user_id=e2f2371c92474e9a905406950c35d170, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.648369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.678519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 643/1299] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:01 2022] PUT /identity/v3/projects/85f464516da748bca9958cd257c72e35/users/e2f2371c92474e9a905406950c35d170/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:01.678851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.680583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.683192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.698617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 657/1300] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:01.699934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.717879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272076a2-4b25-4b2c-9b58-91356256d587 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.767023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:01.852354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.854450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.854685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.855922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d2d1d-7764-4bb1-8fce-04d745a823eb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.856173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.858108 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.860555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.871137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.871845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.872152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.872436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.873100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.879816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.905073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 658/1301] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:01.909138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.921306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 644/1302] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:01.923965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:01.931228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f16d-6427-499a-adbe-5007fb118c34 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.944679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.949134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f89c8-22ef-4b71-9041-a81cf751bb44 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:01.950190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.950561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.950972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] PATH_INFO: `/v3/projects/85f464516da748bca9958cd257c72e35/users/e2f2371c92474e9a905406950c35d170/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.954440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f464516da748bca9958cd257c72e35, user_id=e2f2371c92474e9a905406950c35d170, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.958322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6f50-9353-4f66-bff1-d3cab0ca2bff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:01.964408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:01.965258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:01.965863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:01.965863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] PATH_INFO: `/v3/projects/0dc92d82c0084f94b442c4bf559dc843/users/82bd4e33efca4543b11267457cd2db8a/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:01.995712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 659/1303] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:01 2022] PUT /identity/v3/projects/85f464516da748bca9958cd257c72e35/users/e2f2371c92474e9a905406950c35d170/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:01.996220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc92d82c0084f94b442c4bf559dc843, user_id=82bd4e33efca4543b11267457cd2db8a, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:01.998975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:02.005841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.006182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.006572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.020943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 645/1304] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:01 2022] PUT /identity/v3/projects/0dc92d82c0084f94b442c4bf559dc843/users/82bd4e33efca4543b11267457cd2db8a/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:02.022707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.049238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] MFA Rules not processed for user `e2f2371c92474e9a905406950c35d170`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:02.050586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:02.124630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc3bfd-e251-4488-af16-2ca1b66cc961 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.131108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.131758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.132028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.132376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.133034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:02.134812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:02.217537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 646/1305] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:02.217537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.217537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2e4a-1c26-4e1f-ba9d-62b274079bcc admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.217537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.219008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.219008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.219008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] PATH_INFO: `/v3/projects/0dc92d82c0084f94b442c4bf559dc843/users/82bd4e33efca4543b11267457cd2db8a/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.219008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc92d82c0084f94b442c4bf559dc843, user_id=82bd4e33efca4543b11267457cd2db8a, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:02.219008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1e082-48b6-4df7-8e40-2d162166c205 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:02.219753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 647/1306] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:02 2022] PUT /identity/v3/projects/0dc92d82c0084f94b442c4bf559dc843/users/82bd4e33efca4543b11267457cd2db8a/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:02.228270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.228700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.229043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.274813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 660/1307] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:02.296704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] MFA Rules not processed for user `82bd4e33efca4543b11267457cd2db8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:02.297994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:02.305656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.386638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:02.414928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9455d-de37-4e27-b525-0643cd34f832 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.441368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 648/1308] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:02.480291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.481007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.481398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.481787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.571830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:02.573991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:02.603337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 661/1309] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:02.687549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.699606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5d83-702f-4217-8c97-f67dfc97199b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.705357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.706059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.706345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.706621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.714588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:02.719221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:02.721624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:02.737979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 649/1310] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:02.909065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.936813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:02.944948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72556-1ed7-46f9-b4d5-0a8d8190f309 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.947100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00106e2-77f9-4d08-8e62-45b315c194ea service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:02.960022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.960609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:02.962018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.962319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:02.962696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.963060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:02.963679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:02.963875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:03.113746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:03.114777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:03.116374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15827-c003-4e8e-a339-9ff9665b2343 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:03.118100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747072b1-e367-4f0a-8190-7350646b4b88 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:03.143290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 650/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:03.145740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 662/1312] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:05.980669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1931-b876-4176-ba77-bb71dc82c1fe None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:05.981240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1931-b876-4176-ba77-bb71dc82c1fe None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:05.981240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1931-b876-4176-ba77-bb71dc82c1fe None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:05.983217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 651/1313] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:05.997553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db97de3-f4dd-4e73-a624-fd0fcf27514a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:05.998159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db97de3-f4dd-4e73-a624-fd0fcf27514a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:05.998159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db97de3-f4dd-4e73-a624-fd0fcf27514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:06.064259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5db97de3-f4dd-4e73-a624-fd0fcf27514a None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:06.066648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5db97de3-f4dd-4e73-a624-fd0fcf27514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:06.094824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:06.241534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 663/1314] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:27:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:06.249857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcb4a4-aac5-4c38-b521-fbf814465c1d None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:06.250156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcb4a4-aac5-4c38-b521-fbf814465c1d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:06.250410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcb4a4-aac5-4c38-b521-fbf814465c1d None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:06.251770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 652/1315] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 11 13:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:06.258561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194923-a85d-4a6b-9e2f-7cd08d84d641 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:06.258881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194923-a85d-4a6b-9e2f-7cd08d84d641 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:06.259285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194923-a85d-4a6b-9e2f-7cd08d84d641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:06.298269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-38194923-a85d-4a6b-9e2f-7cd08d84d641 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:06.300285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38194923-a85d-4a6b-9e2f-7cd08d84d641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:06.325705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 664/1316] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:27:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:06.350058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcb4a4-aac5-4c38-b521-fbf814465c1d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:06.377128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcb4a4-aac5-4c38-b521-fbf814465c1d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:06.382951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:06.383182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:06.383761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:06.384206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:06.404768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:06.406370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85f43-df80-4c89-bbc4-3ab0ba591d0b service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:06.431335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 653/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:08.840397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd331d-fd96-4fe0-ba55-8178530dedae None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:08.841030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd331d-fd96-4fe0-ba55-8178530dedae None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:08.841030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd331d-fd96-4fe0-ba55-8178530dedae None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:08.842593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 665/1318] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:08.867744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980621-e70f-4d67-ac73-3a58422bbf1e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:08.869039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980621-e70f-4d67-ac73-3a58422bbf1e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:08.869039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55980621-e70f-4d67-ac73-3a58422bbf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:08.923117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-55980621-e70f-4d67-ac73-3a58422bbf1e None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:08.924388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-55980621-e70f-4d67-ac73-3a58422bbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:08.955072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 654/1319] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:08.962298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89da33-f3a6-4c7d-8dde-3d8239f658de None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:08.962585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89da33-f3a6-4c7d-8dde-3d8239f658de None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:08.962857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89da33-f3a6-4c7d-8dde-3d8239f658de None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:08.964682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 666/1320] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 11 13:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:08.971498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6c6-a8be-467a-bfbc-e66e944e2f90 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:08.971865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6c6-a8be-467a-bfbc-e66e944e2f90 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:08.972230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6c6-a8be-467a-bfbc-e66e944e2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:09.010806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f182d6c6-a8be-467a-bfbc-e66e944e2f90 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:09.012147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f182d6c6-a8be-467a-bfbc-e66e944e2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:09.031221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 655/1321] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:09.047725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89da33-f3a6-4c7d-8dde-3d8239f658de None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:09.065566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89da33-f3a6-4c7d-8dde-3d8239f658de None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:09.071941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:09.072860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:09.073489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:09.074296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:09.097342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:09.099606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:09.136717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 667/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:10.589064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b4eb4-2eb6-4be7-9f8c-816bc684fd46 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:10.589519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b4eb4-2eb6-4be7-9f8c-816bc684fd46 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:10.589614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b4eb4-2eb6-4be7-9f8c-816bc684fd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:10.687866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-312b4eb4-2eb6-4be7-9f8c-816bc684fd46 None None] MFA Rules not processed for user `d439d54a4f2348d78ddb84691f2a4f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:10.689236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-312b4eb4-2eb6-4be7-9f8c-816bc684fd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:10.805754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 656/1323] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:10.825351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:10.853503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43afb5-e63e-4470-b920-2997cb2654db service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:10.862112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:10.863034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:10.863492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:10.864005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:10.887269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:10.889399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:10.901760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 668/1324] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:12.397485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff8e0c-8e0b-4d5b-b571-ade59b726ea4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:12.398327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff8e0c-8e0b-4d5b-b571-ade59b726ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:12.398748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff8e0c-8e0b-4d5b-b571-ade59b726ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:12.430559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7aff8e0c-8e0b-4d5b-b571-ade59b726ea4 None None] MFA Rules not processed for user `2fc3a4b9afd14409b3aef17f49aafd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:12.432107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7aff8e0c-8e0b-4d5b-b571-ade59b726ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:12.489490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:12.524079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 657/1325] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:12.540927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:12.566144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252a816-c6e8-4087-8a98-5e7c99336f8b service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:12.581135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:12.581833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:12.582181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:12.582464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:12.611872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:12.613430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4da21-cfc7-472a-9fe3-cd44732a2cb3 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:12.655336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 669/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:13.937595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65ec82-a479-41fc-bfc1-dabd4d4fa984 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:13.938711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65ec82-a479-41fc-bfc1-dabd4d4fa984 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:13.939265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65ec82-a479-41fc-bfc1-dabd4d4fa984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:13.994663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-eb65ec82-a479-41fc-bfc1-dabd4d4fa984 None None] MFA Rules not processed for user `d439d54a4f2348d78ddb84691f2a4f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:13.997818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-eb65ec82-a479-41fc-bfc1-dabd4d4fa984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:14.018428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 658/1327] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:16.225572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0c0b-be02-4d7c-adc4-9d8179c6d1d4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:16.226895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0c0b-be02-4d7c-adc4-9d8179c6d1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:16.227198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0c0b-be02-4d7c-adc4-9d8179c6d1d4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:16.229029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 670/1328] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 11 13:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:16.236062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacdb-b297-4574-93a5-23eef21467cb None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:16.236629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacdb-b297-4574-93a5-23eef21467cb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:16.236861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacdb-b297-4574-93a5-23eef21467cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:16.368091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5ebdacdb-b297-4574-93a5-23eef21467cb None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:16.369706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5ebdacdb-b297-4574-93a5-23eef21467cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:16.472369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 659/1329] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:27:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:16.500132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe0c0b-be02-4d7c-adc4-9d8179c6d1d4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:16.518726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe0c0b-be02-4d7c-adc4-9d8179c6d1d4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:16.531914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:16.532664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:16.532997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:16.533327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:16.548064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:16.551403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bab841-0d6e-4bcb-84b1-ffc21ed715b8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:16.584129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 671/1330] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:17.422738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:17.429751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:17.430302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:17.518502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] MFA Rules not processed for user `e4e126b005e149258013ce3b6cf15524`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:17.524016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:17.690145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 660/1331] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:17.692815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:17.693417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:17.693417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:17.738289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] MFA Rules not processed for user `e4e126b005e149258013ce3b6cf15524`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:17.739658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:17.751268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 672/1332] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:17.763884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:17.795544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd18ae-ae76-4ead-a91f-8b462dba8cf1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:17.815622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:17.816194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:17.817498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:17.817760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:17.835177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:17.836881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c0348-bb24-4b92-9526-e30f8e848f9f service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:17.863502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:17.864936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 661/1333] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:18.062631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c4468-d90a-4430-a318-9672483d8d68 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:18.077164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:18.079633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:18.080085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:18.080576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:18.086541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:18.110870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:18.113082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3752d41-3eb9-443b-b058-3d1aa9eb49c0 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:18.140333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 673/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:19.992597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:19.993180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:19.993485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:20.112892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] MFA Rules not processed for user `9a4a84c1ffc4466687122a6c67536d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:20.114365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:20.367295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:20.463509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 662/1335] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:20.480928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8545e-94e9-4c68-b551-7a4fbdd817f7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:20.481790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8545e-94e9-4c68-b551-7a4fbdd817f7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:20.482287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8545e-94e9-4c68-b551-7a4fbdd817f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:20.565210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1bb8545e-94e9-4c68-b551-7a4fbdd817f7 None None] MFA Rules not processed for user `2fc3a4b9afd14409b3aef17f49aafd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:20.565210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1bb8545e-94e9-4c68-b551-7a4fbdd817f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:20.565210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 674/1336] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:20.595695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:20.614443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62dea4-2d42-4c6b-ba0b-8fff6701d7bf None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:20.622020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:20.623472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:20.623762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:20.624331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:20.636103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:20.638397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1812e-146d-494a-b3dd-0b6870d032e8 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:20.651331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 663/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:21.991048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:21.991876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:21.992370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.097759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:22.107928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:22.109273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:22.280423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 675/1338] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:22.281231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7973e-9bf1-446a-9042-4f123f29c3bf None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.281719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7973e-9bf1-446a-9042-4f123f29c3bf None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.282678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7973e-9bf1-446a-9042-4f123f29c3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.358932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-69b7973e-9bf1-446a-9042-4f123f29c3bf None None] MFA Rules not processed for user `b50f86c6209f4a728cf3681a7d0a6765`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:22.361536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-69b7973e-9bf1-446a-9042-4f123f29c3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:22.486896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:22.488672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:22.566878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 664/1339] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:22.582207 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9527cf-3d28-4a72-9319-4910661d4d00 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.583468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9527cf-3d28-4a72-9319-4910661d4d00 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.583924 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9527cf-3d28-4a72-9319-4910661d4d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.655173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5c9527cf-3d28-4a72-9319-4910661d4d00 None None] MFA Rules not processed for user `b50f86c6209f4a728cf3681a7d0a6765`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:22.664281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5c9527cf-3d28-4a72-9319-4910661d4d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:22.695710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 665/1340] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:22.698045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a217-618d-4d94-bfdd-5efe2a98fc33 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.698356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a217-618d-4d94-bfdd-5efe2a98fc33 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.698657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288a217-618d-4d94-bfdd-5efe2a98fc33 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.706608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 666/1341] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 11 13:27:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:22.727853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.729091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.729461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.780160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:22.786667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82dfcf-cb8a-4aaa-9b14-47d83e834972 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:22.798812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:22.800711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:22.801748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.802277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.802738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.822835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:22.829316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:22.839956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 667/1342] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:22.844001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 676/1343] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 11 13:27:22.851598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:22.888873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:22.917770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8917dcc-6159-472b-81d8-c66c0016107a service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:22.922623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:22.923573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.924128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.924468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42b77-5465-49d9-8fad-0576ecb3e52e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:22.924792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.942854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:22.950480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:22.952769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:22.953317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:22.954377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:22.957508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99374b2-4daf-4aba-b677-b905d2aa5d12 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:22.982331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:22.987656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:22.988208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 677/1344] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:23.006094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 668/1345] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:23.864582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f40d0-1776-41d3-b0d1-8ff8de61107b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:23.866713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f40d0-1776-41d3-b0d1-8ff8de61107b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:23.868806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f40d0-1776-41d3-b0d1-8ff8de61107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:23.962563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0a6f40d0-1776-41d3-b0d1-8ff8de61107b None None] MFA Rules not processed for user `9a4a84c1ffc4466687122a6c67536d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:23.970875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0a6f40d0-1776-41d3-b0d1-8ff8de61107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:24.037772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 678/1346] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:24.389875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:24.677622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31149326-de75-48ec-a582-6f68c83492ce service nova] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:24.693916 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:24.697697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:24.700064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:24.700425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] PATH_INFO: `/v3/users/e352807f0fed42a7adc6985a43028dab` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:24.704002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e352807f0fed42a7adc6985a43028dab)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:24.707549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: WARNING py.warnings [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:27:24.707549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: warnings.warn(msg) Oct 11 13:27:24.707549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: Oct 11 13:27:24.708856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:24.852331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:24.901419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e352807f0fed42a7adc6985a43028dab'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:24.903990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e352807f0fed42a7adc6985a43028dab'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:24.968996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e352807f0fed42a7adc6985a43028dab'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:24.986545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e352807f0fed42a7adc6985a43028dab'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:24.987303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e352807f0fed42a7adc6985a43028dab'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:25.025060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 669/1347] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:24 2022] DELETE /identity/v3/users/e352807f0fed42a7adc6985a43028dab => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:25.029556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4a43-b118-4f22-9764-72f88a25aa1c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:25.030068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4a43-b118-4f22-9764-72f88a25aa1c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:25.030461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4a43-b118-4f22-9764-72f88a25aa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:25.076964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3bda4a43-b118-4f22-9764-72f88a25aa1c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:25.078363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3bda4a43-b118-4f22-9764-72f88a25aa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:25.237134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:25.265623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 679/1348] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:25.350978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:25.353184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:25.630383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801fd7-e85c-44d8-b174-4c7c9df3c359 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:25.643607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:25.646945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:25.647239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:25.647518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:25.692969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:25.701366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:25.732590 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 670/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:26.252795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:26.254176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:26.255368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:26.352295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:26.362648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:26.410652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 680/1350] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 11 13:27:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:26.507802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:26.511745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:26.566598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938550-62d6-428a-a06d-05fc3e76ebfc service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:26.619298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:26.623789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:26.625598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:26.626903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:26.668600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:26.679087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:26.717503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 671/1351] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:27.441206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:27.444250 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:27.474692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76b73-3c62-45b8-8a62-09ded775070c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:27.489226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:27.491409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:27.491941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:27.492573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] PATH_INFO: `/v3/projects/266ea3e1b3fa4c76ab08cb65dedbb9b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:27.497323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=266ea3e1b3fa4c76ab08cb65dedbb9b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:27.498675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: WARNING py.warnings [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 13:27:27.498675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: warnings.warn(msg) Oct 11 13:27:27.498675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: Oct 11 13:27:27.504328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:27.662413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266ea3e1b3fa4c76ab08cb65dedbb9b4', 'request_id': 'req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:27.702790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266ea3e1b3fa4c76ab08cb65dedbb9b4', 'request_id': 'req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:27.720617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266ea3e1b3fa4c76ab08cb65dedbb9b4', 'request_id': 'req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:27.750858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] The token cache is being invalidate because project 266ea3e1b3fa4c76ab08cb65dedbb9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:27.751263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:27.753071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8a7e4d0e-6d6a-4757-9d81-91e0e7ece0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:27.757155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 681/1352] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:27 2022] DELETE /identity/v3/projects/266ea3e1b3fa4c76ab08cb65dedbb9b4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:27.762439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:27.792033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20027da-2323-42d5-a6aa-18110e3c4692 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:28.107314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:28.107982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:28.108295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:28.108570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] PATH_INFO: `/v3/users/e2f2371c92474e9a905406950c35d170` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:28.110821 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f2371c92474e9a905406950c35d170)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:28.112864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:28.218668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:28.244057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f2371c92474e9a905406950c35d170'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:28.244865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f2371c92474e9a905406950c35d170'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:28.267439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f2371c92474e9a905406950c35d170'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:28.287275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f2371c92474e9a905406950c35d170'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:28.288216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f2371c92474e9a905406950c35d170'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:28.317151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 672/1353] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:27 2022] DELETE /identity/v3/users/e2f2371c92474e9a905406950c35d170 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:28.758675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26e369-0aee-4f5c-b547-414c416a7641 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:28.759651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26e369-0aee-4f5c-b547-414c416a7641 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:28.761620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26e369-0aee-4f5c-b547-414c416a7641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:28.825251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ae26e369-0aee-4f5c-b547-414c416a7641 None None] MFA Rules not processed for user `b50f86c6209f4a728cf3681a7d0a6765`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:28.826493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ae26e369-0aee-4f5c-b547-414c416a7641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:28.979255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 682/1354] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:29.396375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:29.398308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:29.606053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f3dc-b76f-4c9f-ab1d-ff2df8fb931f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:29.638518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:29.640255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:29.640639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:29.640923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] PATH_INFO: `/v3/users/5734434bbd3348319c23c8055ec2f0ac` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:29.651165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5734434bbd3348319c23c8055ec2f0ac)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:29.658787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:29.794513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5734434bbd3348319c23c8055ec2f0ac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:29.795621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5734434bbd3348319c23c8055ec2f0ac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:29.845944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5734434bbd3348319c23c8055ec2f0ac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:29.882416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5734434bbd3348319c23c8055ec2f0ac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:29.884774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5734434bbd3348319c23c8055ec2f0ac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:29.958613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 673/1355] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:29 2022] DELETE /identity/v3/users/5734434bbd3348319c23c8055ec2f0ac => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:29.962833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617fd7-0820-4091-8d44-bedc4abf62b7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:29.963980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617fd7-0820-4091-8d44-bedc4abf62b7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:29.964431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617fd7-0820-4091-8d44-bedc4abf62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:30.002113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d1617fd7-0820-4091-8d44-bedc4abf62b7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:30.018587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d1617fd7-0820-4091-8d44-bedc4abf62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:30.284349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 683/1356] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:30.313332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:30.315150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:30.445105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:30.492888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebebbd6-883e-4ba2-be92-9189cb891d7b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:30.514672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:30.516024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:30.516600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:30.517018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:30.552845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:30.568413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb406-626c-4902-a025-f8d80e6732a0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:30.593945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 674/1357] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:30.637877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1835e94-190e-4fe0-a686-914e69ca40ae None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:30.638473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1835e94-190e-4fe0-a686-914e69ca40ae None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:30.638937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1835e94-190e-4fe0-a686-914e69ca40ae None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:30.640684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 684/1358] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:30.926530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011683c-d92d-4abf-a147-a6ca589e6f88 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:30.926891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011683c-d92d-4abf-a147-a6ca589e6f88 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:30.927077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011683c-d92d-4abf-a147-a6ca589e6f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:30.997097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5011683c-d92d-4abf-a147-a6ca589e6f88 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:31.005624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5011683c-d92d-4abf-a147-a6ca589e6f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:31.288487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 675/1359] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:27:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:32.555229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:32.555804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:32.555804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:32.557682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 685/1360] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:32.582371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:32.582932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:32.583542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:32.646241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:32.650739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:32.687305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 676/1361] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:34.306652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:34.309157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:34.336455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a81139-9cc5-4699-84ef-9e82445c8ff4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:34.345621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:34.348135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:34.349542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:34.350092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:34.361685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:34.364040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:34.388539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:34.390585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a1d88-f2d8-461e-9b24-1e725158a850 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:34.396485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703d42-75de-49c2-a7f7-662222150431 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:34.409533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:34.413009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:34.413985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:34.414970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] PATH_INFO: `/v3/projects/85f464516da748bca9958cd257c72e35` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:34.417277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f464516da748bca9958cd257c72e35)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:34.423105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:34.428712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 686/1362] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:34.648980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f464516da748bca9958cd257c72e35', 'request_id': 'req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:34.692734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f464516da748bca9958cd257c72e35', 'request_id': 'req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:34.716811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f464516da748bca9958cd257c72e35', 'request_id': 'req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:34.770897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] The token cache is being invalidate because project 85f464516da748bca9958cd257c72e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:34.775887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:34.778378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:34.805166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 677/1363] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:34 2022] DELETE /identity/v3/projects/85f464516da748bca9958cd257c72e35 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:34.806852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:34.807142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:34.807592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:34.822610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:34.898359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:34.899956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:34.936803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:34.938073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.149365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:35.176543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 687/1364] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:35.177571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:35.182605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.233391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0251-5ef7-4867-8428-e3e7c21de349 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:35.253953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:35.254641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.254924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.255188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] PATH_INFO: `/v3/projects/b225bb837bdd4e6390c7b24cb239fc1d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.266910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b225bb837bdd4e6390c7b24cb239fc1d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:35.277036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:35.374295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6314b-8817-4f3a-a818-ba66abd3a5f3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:35.413719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:35.417884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.419563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.422427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.430730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.456532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b225bb837bdd4e6390c7b24cb239fc1d', 'request_id': 'req-66747665-3ec9-4cfe-b078-48fb8545657b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:35.481137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b225bb837bdd4e6390c7b24cb239fc1d', 'request_id': 'req-66747665-3ec9-4cfe-b078-48fb8545657b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:35.497700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b225bb837bdd4e6390c7b24cb239fc1d', 'request_id': 'req-66747665-3ec9-4cfe-b078-48fb8545657b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:35.507052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:35.513796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fefb18-734e-4ffc-a8a2-70f84bce4b1b service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:35.529673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] The token cache is being invalidate because project b225bb837bdd4e6390c7b24cb239fc1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:35.529978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:35.535018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-66747665-3ec9-4cfe-b078-48fb8545657b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:35.540471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 688/1365] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:35 2022] DELETE /identity/v3/projects/b225bb837bdd4e6390c7b24cb239fc1d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:35.541799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.542081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.548778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.559827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 678/1366] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:35.562882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84f7f-44b9-4e3e-894c-13bcdebd0d16 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.563186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84f7f-44b9-4e3e-894c-13bcdebd0d16 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.566070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c84f7f-44b9-4e3e-894c-13bcdebd0d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.627106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:35.630719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.640670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-50c84f7f-44b9-4e3e-894c-13bcdebd0d16 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:35.644811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-50c84f7f-44b9-4e3e-894c-13bcdebd0d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.801356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 679/1367] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:35.803111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.803401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.803802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.844583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:35.846929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:35.859633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 689/1368] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:35.864514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:35.874077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 680/1369] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:35.880328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:35.909743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6769e-1c13-4d5a-948d-4ce8ac7b39a9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:35.916280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7b949-6625-491a-84b7-71633a464930 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:35.937890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:35.938551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.938827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.939109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.939777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:27:35.940099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:35.941847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:35.944889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:35.947577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:35.947808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:35.948089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:35.950951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:27:35.951306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:35.953009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.027788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 681/1370] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:27:36.030234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.043478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 690/1371] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:27:36.045380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.080738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd448-a55d-40f6-9980-8a71b514482b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.093282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.093924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.094203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.094473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.097667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924cf8f-b6db-49c3-83d8-91f66ea7f999 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.106044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.132602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.140793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.140793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.140793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.140793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.141380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.151028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.180316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:36.254584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 691/1372] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:36 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:36.269116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.291627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 682/1373] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:36 2022] POST /identity/v3/projects => generated 403 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:36.292804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.517233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed23cec-44f0-4e14-aef5-ece67d2a9e8d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.555013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2fbf6-15b8-41da-b885-4331cf3fb574 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.567130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.573152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.573152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.573152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.573152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.575591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.632947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 683/1374] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:36 2022] POST /identity/v3/users => generated 302 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:36.634351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.661742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658285c0-3eac-4bf4-b3e7-7f8a401f50b6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.671655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.672555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.678184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.678184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.678184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.679213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.728884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 684/1375] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:36.733891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.772179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013af0e-b8d2-4da7-beed-1565fa898e3b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.792061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.797057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.797414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.797908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.799056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.810846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.823411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.826194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.828168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:36.843338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.845736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.850270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.858713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.860759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 685/1376] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:36.861699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.886487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782284a8-b6f4-4e81-b583-4d4a11e7a95f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.901097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.904831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.905126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.905406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] PATH_INFO: `/v3/projects/4f8581b4b5b740689968e8d4ba8d5f1b/users/493f57dfe68f4eb2b05d8f675feaec80/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.922862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 692/1377] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:36 2022] POST /identity/v3/users => generated 285 bytes in 655 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:36.927001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:36.949979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1c4c-aa4c-4ea4-8793-0f11a0c53e36 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:36.950190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8581b4b5b740689968e8d4ba8d5f1b, user_id=493f57dfe68f4eb2b05d8f675feaec80, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.961065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:36.961566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.962326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:36.962836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:36.963301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:36.964342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:36.968671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35dae4-b1a0-4706-8103-0ffbe1dc6fd8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:36.992608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 686/1378] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:36 2022] PUT /identity/v3/projects/4f8581b4b5b740689968e8d4ba8d5f1b/users/493f57dfe68f4eb2b05d8f675feaec80/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:36.997444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 693/1379] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:36.998715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.004199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.004755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.005176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.069836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] MFA Rules not processed for user `493f57dfe68f4eb2b05d8f675feaec80`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:37.072050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:37.163827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d18711-41ba-4cc0-87fa-72bfe307e8d3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.175706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.176382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.176702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.176937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.177602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.179322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.211019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 687/1380] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:37.211950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.237504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 694/1381] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:37.241370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03efff7-bbee-4d01-ba05-c3a2fabe29ab admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.241982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.248325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.251856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.252167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.252458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] PATH_INFO: `/v3/projects/cfa470ef191e48b58f912b16e452dd17/users/16fa913ca91f4a79801fee40d53e1b3b/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.261366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063b97b-e2f6-4dc5-829c-93465176d212 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.275112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.277559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa470ef191e48b58f912b16e452dd17, user_id=16fa913ca91f4a79801fee40d53e1b3b, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.278223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.278607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.278930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.279657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.281715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.290712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f256ac9-ad11-4f58-a26f-e7b5ef1aed9e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.296687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:37.327310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 688/1382] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:37 2022] PUT /identity/v3/projects/cfa470ef191e48b58f912b16e452dd17/users/16fa913ca91f4a79801fee40d53e1b3b/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:37.338694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.339959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.340754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.359626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 695/1383] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:37.360635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.425268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] MFA Rules not processed for user `16fa913ca91f4a79801fee40d53e1b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:37.427219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:37.652056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a51246-538f-4873-89f6-600d7f834fd0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.671715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.672683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.673271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.673740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.674580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.680987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.728373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 689/1384] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:37.733386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.758903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414c16-b08a-4423-96b0-cac4ccab06ea None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.761999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 696/1385] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/users => generated 308 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:37.763146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.775518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.776303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.776591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.776856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.777540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.779824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.800939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b12f-4f2d-4978-a623-532c19b9fca3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.825275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:37.827921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:37.828330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:37.828984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:37.834189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:37.836217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:37.856032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 690/1386] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:27:37.859491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 697/1387] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:37.860944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.862914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:37.889761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75b96-2efc-440a-a8b2-eebb2db0292c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:37.907414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:38.077302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe896249-4829-487d-be34-bf6b4b193918 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.093952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.100665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.100665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.101556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.102358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.106516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.111589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.111980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.115636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.116222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] PATH_INFO: `/v3/projects/344b775e7e1c45e5b564ac8b9367ec36/users/f80aabaa9f284679adc889870cd07fd8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.170231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344b775e7e1c45e5b564ac8b9367ec36, user_id=f80aabaa9f284679adc889870cd07fd8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.177251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.210587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 691/1388] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:37 2022] POST /identity/v3/users => generated 290 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:38.211418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.215683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 698/1389] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:37 2022] PUT /identity/v3/projects/344b775e7e1c45e5b564ac8b9367ec36/users/f80aabaa9f284679adc889870cd07fd8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:38.217154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.256935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7e0f-0a86-40fe-bdf8-684c7bbc713d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.378266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:38.428193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e059-0e2e-4fd5-9c34-93169921e906 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.444511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.446356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.447159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.447159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.447658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.451610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.461877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.464632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.464958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.465298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.465896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.467519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.477774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 699/1390] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:38.481572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.501521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 692/1391] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:38.507086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.513120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e98f-a773-426a-9860-f3bd5b97d0a4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.528179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.531619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.531982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.532352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] PATH_INFO: `/v3/projects/344b775e7e1c45e5b564ac8b9367ec36/users/f80aabaa9f284679adc889870cd07fd8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.535201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4f04-8811-47ef-a888-15c5259d9676 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.536924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344b775e7e1c45e5b564ac8b9367ec36, user_id=f80aabaa9f284679adc889870cd07fd8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.542865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c34b7-f461-40bd-82d3-447447304141 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.544963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.545782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.546127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.546471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] PATH_INFO: `/v3/projects/16eb334a28d14759bcb464150e313cb8/users/be0da48ee483404fa5f6c1c95dd20852/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.567283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16eb334a28d14759bcb464150e313cb8, user_id=be0da48ee483404fa5f6c1c95dd20852, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.573066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.578795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 700/1392] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:38 2022] PUT /identity/v3/projects/344b775e7e1c45e5b564ac8b9367ec36/users/f80aabaa9f284679adc889870cd07fd8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:38.595904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.596470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.596866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.603128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 693/1393] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:38 2022] PUT /identity/v3/projects/16eb334a28d14759bcb464150e313cb8/users/be0da48ee483404fa5f6c1c95dd20852/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:38.607057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.635794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] MFA Rules not processed for user `f80aabaa9f284679adc889870cd07fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:38.638248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:38.700462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907e7b-a406-4fc3-9765-eeb11430e84f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.712080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.712870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.713370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.713721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.714446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.718095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.740104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 694/1394] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:38.745338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.760304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcad90-1387-4447-aad4-6640ca3641fb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.767385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:38.769377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.770302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.770975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] PATH_INFO: `/v3/projects/16eb334a28d14759bcb464150e313cb8/users/be0da48ee483404fa5f6c1c95dd20852/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.773474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] RBAC: Authorizing `identity:create_grant(project_id=16eb334a28d14759bcb464150e313cb8, user_id=be0da48ee483404fa5f6c1c95dd20852, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:38.776751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b5ec4-5106-4f87-b7f5-fc0a433e40be admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:38.785621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:38.789825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 701/1395] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:38.810193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 695/1396] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:38 2022] PUT /identity/v3/projects/16eb334a28d14759bcb464150e313cb8/users/be0da48ee483404fa5f6c1c95dd20852/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:38.810933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:38.812152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:38.812388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:38.812631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:38.813006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:38.855294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] MFA Rules not processed for user `be0da48ee483404fa5f6c1c95dd20852`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:38.856769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:38.953873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04ca33-3389-4a65-aad8-ef97d25b28d6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:38.988589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 696/1397] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:39.001303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:39.003172 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:39.042964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:39.043696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:39.043964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:39.044330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:39.084850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:39.115705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b94d-af1f-49a7-8f9e-3661a4469f27 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:39.119896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:39.120662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:39.120945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:39.121217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:39.131764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:39.133043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:39.143534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:39.144269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 697/1398] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:39.146522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:39.160480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 702/1399] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:38 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:39.223069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:39.246172 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bf48a-da28-41ec-94fe-7546c8408d2b service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:39.250082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:39.250687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:39.250964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:39.251240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:39.323662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:39.325799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f0b42-d62c-403c-bdc0-3ae371e3d193 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:39.337403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 698/1400] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:42.455270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:42.525485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b357-def7-40cd-86c0-4949de42c1cd service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:42.530690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:42.531343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:42.531617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:42.531882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] PATH_INFO: `/v3/users/493f57dfe68f4eb2b05d8f675feaec80` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:42.533132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493f57dfe68f4eb2b05d8f675feaec80)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:42.535112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:42.598305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493f57dfe68f4eb2b05d8f675feaec80'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.605669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493f57dfe68f4eb2b05d8f675feaec80'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.619127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493f57dfe68f4eb2b05d8f675feaec80'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.619899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493f57dfe68f4eb2b05d8f675feaec80'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.628808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493f57dfe68f4eb2b05d8f675feaec80'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.635292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 703/1401] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:42 2022] DELETE /identity/v3/users/493f57dfe68f4eb2b05d8f675feaec80 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:42.639904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:42.640390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:42.640480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:42.660027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:42.661402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:42.682228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:42.755315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 699/1402] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:42.766751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:42.768725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:42.825160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:42.843958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69cc91-5f27-459e-a0eb-06a621dd9560 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:42.846570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:42.847979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:42.848544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:42.849371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:42.849819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:42.850306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:42.860792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:42.860937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a947f6-c731-4a9d-9fae-32b5ab67dd2d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:42.862813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a19df-7b6d-4ee3-a3d7-b7c9f288d748 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:42.865905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:42.866548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:42.866895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:42.867164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] PATH_INFO: `/v3/users/16fa913ca91f4a79801fee40d53e1b3b` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:42.868705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16fa913ca91f4a79801fee40d53e1b3b)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:42.871142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:42.876351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 704/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:42.926026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16fa913ca91f4a79801fee40d53e1b3b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.926722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16fa913ca91f4a79801fee40d53e1b3b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.940540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16fa913ca91f4a79801fee40d53e1b3b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.949486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16fa913ca91f4a79801fee40d53e1b3b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.950348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16fa913ca91f4a79801fee40d53e1b3b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:42.966994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 700/1404] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:42 2022] DELETE /identity/v3/users/16fa913ca91f4a79801fee40d53e1b3b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:42.972576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:42.972867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:42.973141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:42.993346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:42.994561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.070679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 705/1405] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:43.081873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.083328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.146501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b150b-2872-4a6b-9432-f7010e7f60a5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:43.150561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:43.151183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:43.151459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:43.151809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:43.163153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:43.164748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:43.177426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 701/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:43.202047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.203453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.216967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad15e30-a811-4bd9-a9ac-04366eee23f0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:43.225767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:43.226856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:43.227311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:43.228210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] PATH_INFO: `/v3/projects/4f8581b4b5b740689968e8d4ba8d5f1b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:43.229794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8581b4b5b740689968e8d4ba8d5f1b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:43.233207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:43.298176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8581b4b5b740689968e8d4ba8d5f1b', 'request_id': 'req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.304878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8581b4b5b740689968e8d4ba8d5f1b', 'request_id': 'req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.314011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8581b4b5b740689968e8d4ba8d5f1b', 'request_id': 'req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.331575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] The token cache is being invalidate because project 4f8581b4b5b740689968e8d4ba8d5f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:43.332146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.332928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.334787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 706/1407] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:43 2022] DELETE /identity/v3/projects/4f8581b4b5b740689968e8d4ba8d5f1b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:43.338538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.348725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5344da6-5a10-4187-8c49-d057d7177145 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:43.401906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:43.425037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:43.425697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:43.426156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:43.426432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] PATH_INFO: `/v3/users/f80aabaa9f284679adc889870cd07fd8` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:43.427657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80aabaa9f284679adc889870cd07fd8)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:43.429724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:43.490358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80aabaa9f284679adc889870cd07fd8'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.491067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80aabaa9f284679adc889870cd07fd8'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.506476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80aabaa9f284679adc889870cd07fd8'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.515125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80aabaa9f284679adc889870cd07fd8'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.515932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80aabaa9f284679adc889870cd07fd8'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.530014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 702/1408] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:43 2022] DELETE /identity/v3/users/f80aabaa9f284679adc889870cd07fd8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:43.693184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.694831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.711210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18ec2-cda4-4e07-a5c4-b3c71340cb14 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:43.799927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:43.800842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:43.801219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:43.801498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] PATH_INFO: `/v3/projects/cfa470ef191e48b58f912b16e452dd17` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:43.802652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa470ef191e48b58f912b16e452dd17)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:43.804886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:43.813497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:43.866379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.868799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.878397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa470ef191e48b58f912b16e452dd17', 'request_id': 'req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.883425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db9f61-3061-4b81-96ba-db65b8ecd842 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:43.896305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa470ef191e48b58f912b16e452dd17', 'request_id': 'req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.910545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa470ef191e48b58f912b16e452dd17', 'request_id': 'req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.924945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] The token cache is being invalidate because project cfa470ef191e48b58f912b16e452dd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:43.925278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.925984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:43.927767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 707/1409] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:43 2022] DELETE /identity/v3/projects/cfa470ef191e48b58f912b16e452dd17 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:43.932303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:43.933204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:43.971030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:43.971861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:43.972169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:43.972445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] PATH_INFO: `/v3/projects/344b775e7e1c45e5b564ac8b9367ec36` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:43.974963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=344b775e7e1c45e5b564ac8b9367ec36)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:43.977537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.010356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a46e-b5df-4b19-89cc-901a2eccd6b3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.016015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.016836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.017103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.017821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] PATH_INFO: `/v3/users/be0da48ee483404fa5f6c1c95dd20852` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.019036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0da48ee483404fa5f6c1c95dd20852)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.021207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.035386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344b775e7e1c45e5b564ac8b9367ec36', 'request_id': 'req-27ac6f72-da03-402c-8d67-d5f91877bc2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.044015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344b775e7e1c45e5b564ac8b9367ec36', 'request_id': 'req-27ac6f72-da03-402c-8d67-d5f91877bc2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.053932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344b775e7e1c45e5b564ac8b9367ec36', 'request_id': 'req-27ac6f72-da03-402c-8d67-d5f91877bc2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.057100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:44.068111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] The token cache is being invalidate because project 344b775e7e1c45e5b564ac8b9367ec36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:44.068863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.070364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-27ac6f72-da03-402c-8d67-d5f91877bc2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.077390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 703/1410] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:43 2022] DELETE /identity/v3/projects/344b775e7e1c45e5b564ac8b9367ec36 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:44.078935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.079365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.079776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.081192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0da48ee483404fa5f6c1c95dd20852'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.090484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0da48ee483404fa5f6c1c95dd20852'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.102118 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:44.103405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0da48ee483404fa5f6c1c95dd20852'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.103560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:44.104295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0da48ee483404fa5f6c1c95dd20852'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.113369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d1cafe3d-18f4-4bf2-a6c8-ae5003988a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0da48ee483404fa5f6c1c95dd20852'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:44.122140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 708/1411] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:43 2022] DELETE /identity/v3/users/be0da48ee483404fa5f6c1c95dd20852 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:44.192710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 704/1412] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:44.201824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.202216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.202554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.224947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:44.226153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:44.237164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:44.351021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.351956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 709/1413] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:44.365733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9af08-ee85-44a2-b26a-909126148ca3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.371581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.372395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.372671 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.372937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.373787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:27:44.374093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.375686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.395374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 705/1414] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:27:44.400980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.411965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3227-97c1-4ff0-8526-72b067704f32 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.418345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.418707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.418979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.419583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.420095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.422536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.453167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 710/1415] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:44.457921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.470440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae9e6-4c7e-4581-861d-a690787bb35d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.546147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.546776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.547098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.547372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.548093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.550331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.581354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 706/1416] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:44.590925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.603889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba158b03-7644-4b14-8017-0cbe3fe21eb7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.609390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.610157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.610470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.610792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.611460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.613447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.632788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 711/1417] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:44.634387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.648562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f667f-d758-4dee-a47e-9b1bd5353228 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.654793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.655431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.655706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.656486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.656667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.658809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe2402-fe37-4bee-a028-2bf4e72daef1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.672294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 707/1418] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:44.677764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.693605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747efa9-4f81-47d1-a405-9eeca9752638 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:44.701296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:44.702633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.702911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.703511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] PATH_INFO: `/v3/projects/12f28bd3acf348b1923cedbbc6e47edd/users/7c6bdec02848467c9ccb30965b19b63f/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.734677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f28bd3acf348b1923cedbbc6e47edd, user_id=7c6bdec02848467c9ccb30965b19b63f, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:44.737760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:44.756167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 712/1419] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:44 2022] PUT /identity/v3/projects/12f28bd3acf348b1923cedbbc6e47edd/users/7c6bdec02848467c9ccb30965b19b63f/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:44.765142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:44.765494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:44.765837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:44.810128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] MFA Rules not processed for user `7c6bdec02848467c9ccb30965b19b63f`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:44.813071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:44.891073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:44.943607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:44.944143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 708/1420] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:45.071517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb020c1-8cd8-4426-b5ca-acc601e74481 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.084955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.085894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.086376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.087051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.088049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.090503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.139420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 713/1421] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:44 2022] POST /identity/v3/projects => generated 355 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:27:45.143680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.156099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e984a14-a05b-40ac-9b6c-8674b9d8ad8e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.269382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.270022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.270295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.270562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.271209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.273175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.311398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.323065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 709/1422] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:45.326027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fa8e-6ade-4f02-ad39-f59b253f2e3b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.339669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.340655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.340950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.341471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.342234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.345878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.354222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.356252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.373428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 714/1423] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:45.377401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece46e2-3a5d-40e4-9e5f-71ed7b93ef87 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.383483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.389018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.389157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.389589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.389886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] PATH_INFO: `/v3/projects/16eb334a28d14759bcb464150e313cb8` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.392745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16eb334a28d14759bcb464150e313cb8)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.394986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.401158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feb77-6efd-416f-a8df-c1e06fa2a804 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.409865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.411525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.412004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.412947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] PATH_INFO: `/v3/projects/59014cac09284fd4ac87ef28777787f2/users/39d6b1fd0101427d8c038a6a64d0af7c/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.436672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59014cac09284fd4ac87ef28777787f2, user_id=39d6b1fd0101427d8c038a6a64d0af7c, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.440922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.460694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 715/1424] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:45 2022] PUT /identity/v3/projects/59014cac09284fd4ac87ef28777787f2/users/39d6b1fd0101427d8c038a6a64d0af7c/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:45.465181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.470711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16eb334a28d14759bcb464150e313cb8', 'request_id': 'req-3a53dbfc-56de-46fd-b508-75744428e63d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:45.479652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:45.480831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16eb334a28d14759bcb464150e313cb8', 'request_id': 'req-3a53dbfc-56de-46fd-b508-75744428e63d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:45.492186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16eb334a28d14759bcb464150e313cb8', 'request_id': 'req-3a53dbfc-56de-46fd-b508-75744428e63d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:45.507064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] The token cache is being invalidate because project 16eb334a28d14759bcb464150e313cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:45.507741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:45.508463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a53dbfc-56de-46fd-b508-75744428e63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:45.512393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 710/1425] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:45 2022] DELETE /identity/v3/projects/16eb334a28d14759bcb464150e313cb8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:45.516819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b920f2-20b4-4361-8df3-edeeb6899789 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.517106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b920f2-20b4-4361-8df3-edeeb6899789 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.517691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b920f2-20b4-4361-8df3-edeeb6899789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.542425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b9b920f2-20b4-4361-8df3-edeeb6899789 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:45.545540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b9b920f2-20b4-4361-8df3-edeeb6899789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.561006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b0b3-634a-4ea6-b574-56ee701f8672 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.613580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:45.628171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 711/1426] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:45.632588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.632770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.632896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.633127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.633413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.633720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.634026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.634704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.636370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.651816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 716/1427] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:45.652471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:45.653697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.655668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.657297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.661679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 712/1428] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:45.667596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.669584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7e03-14c7-48f7-8801-ed2d73cc77e6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.674317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.675566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.675851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.676141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] PATH_INFO: `/v3/projects/59014cac09284fd4ac87ef28777787f2/users/39d6b1fd0101427d8c038a6a64d0af7c/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.678076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59014cac09284fd4ac87ef28777787f2, user_id=39d6b1fd0101427d8c038a6a64d0af7c, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.679250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b232c-f0db-4f79-a23e-0c525119249d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.680824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4395e26-dcac-48a8-b726-1bac2f7c3649 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.684518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.685123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.685399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.685673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.686738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:27:45.686738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.689464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.693358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 717/1429] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:45 2022] PUT /identity/v3/projects/59014cac09284fd4ac87ef28777787f2/users/39d6b1fd0101427d8c038a6a64d0af7c/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:45.701354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 713/1430] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:27:45.705562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.706716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.707204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.707278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.718011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78c5e-c700-498e-8a22-c60020f6a971 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.743523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] MFA Rules not processed for user `39d6b1fd0101427d8c038a6a64d0af7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:45.744809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.813326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.814308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.815468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.815935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.817142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.819207 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:45.857502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 718/1431] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:45.859648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 714/1432] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:27:45.860257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.883401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:45.885276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:45.903786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ccff5-7292-4f36-899f-9dbb65a3b9a4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.926651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:45.949307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c32b37-345f-4abc-b337-8f171d898c55 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:45.953201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.953885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.954165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.954434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.975293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:45.975909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:45.976522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:45.976796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:45.977746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:45.979798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.006627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 719/1433] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:45 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:46.011662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.022816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12309c-bffb-4d35-8f41-fb6cfe980517 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.027622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.028767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.029388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.029715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.029831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.030054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.030712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.032353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.041206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 715/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:27:46.046949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 720/1435] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:46.048033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.065622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac7ec5-9392-4bbd-8159-ab8a6f668edf service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.070261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.070869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.071140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.072250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.072470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.074250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.095333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.098041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 716/1436] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:46.109923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2ac0b-d581-4f3d-9a45-9b2f67be28e0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.115201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.115987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.116427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.116804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] PATH_INFO: `/v3/projects/5d2a195279184b52875bfee8c9aa5ac6/users/87ac7ab1d3ab4611a8a5729e4019b7ff/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.146213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2a195279184b52875bfee8c9aa5ac6, user_id=87ac7ab1d3ab4611a8a5729e4019b7ff, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.146755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.149067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a6a9-cabb-4238-ae87-2ad40b2f2a28 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.154125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:46.158497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade25f12-5275-4357-97d2-b6430db2a61c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.164185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.165038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.165409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.165769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.167594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:46.172145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 721/1437] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:46 2022] PUT /identity/v3/projects/5d2a195279184b52875bfee8c9aa5ac6/users/87ac7ab1d3ab4611a8a5729e4019b7ff/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:46.182909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.183201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.183476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.212947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] MFA Rules not processed for user `87ac7ab1d3ab4611a8a5729e4019b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:46.214076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:46.250884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.252509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.263445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 717/1438] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:46.300720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 722/1439] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:46.304620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.325219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309893d9-0b49-4078-97bb-5d47067df45d service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.398100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.398775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.399056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.399326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.400054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.402647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.442957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 718/1440] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:46 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:46.447076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.459057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fad4e-a7ee-4f6c-9241-d94696f20768 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.524429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.525049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.525340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.525604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.526232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.528150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.544936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:46.554524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 723/1441] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:46 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:46.555738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.566362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04742c-2bad-4052-a300-c804c53a9d21 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.571477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.572091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.572410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.572706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.573329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.574929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.587851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 719/1442] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:27:46.592395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.602460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688e7f6-1036-43d0-b073-fbcaa6ba9ea5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.607324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.607979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.608303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.608575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] PATH_INFO: `/v3/projects/5304abea04954aea87c5a4225b8c0db6/users/da19a9fdbda54991ad980166232fb59f/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.623220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5304abea04954aea87c5a4225b8c0db6, user_id=da19a9fdbda54991ad980166232fb59f, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.625929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.639488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 724/1443] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:46 2022] PUT /identity/v3/projects/5304abea04954aea87c5a4225b8c0db6/users/da19a9fdbda54991ad980166232fb59f/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:46.643971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.654841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008c925-8000-4075-aa80-6a514bc4c9d0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.713351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.713974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.714256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.714528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.715180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.716824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e336c3f-f206-4ec5-b705-81507a2e289d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.726659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 720/1444] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:46.731243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.742927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a7870-ce23-4ef1-9dd7-df738572c14e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.748933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.749601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.749875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.750141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] PATH_INFO: `/v3/projects/5304abea04954aea87c5a4225b8c0db6/users/da19a9fdbda54991ad980166232fb59f/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.752536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5304abea04954aea87c5a4225b8c0db6, user_id=da19a9fdbda54991ad980166232fb59f, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.755183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.767067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 725/1445] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:46 2022] PUT /identity/v3/projects/5304abea04954aea87c5a4225b8c0db6/users/da19a9fdbda54991ad980166232fb59f/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:46.778940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.779228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.779504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.806849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:46.810329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] MFA Rules not processed for user `da19a9fdbda54991ad980166232fb59f`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:46.812415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:46.887389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 721/1446] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:46.901376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:46.903405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:46.966202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30206-a09e-48d2-ac7b-e8b61faf1251 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:46.970057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:46.970683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:46.970962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:46.971233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:46.980350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:46.981790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:46.991877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 726/1447] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:47.078209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:47.088183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04fa3-b9b7-4d9e-a144-9fc3588ca39a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:47.092181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:47.092874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:47.093241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:47.093568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:47.150637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:47.152753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29aa6b0-8801-446f-9609-a833ce329aff service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:47.162108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 722/1448] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:53.488818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:53.552597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaf24d-4d34-4b51-a6a6-4260578578d7 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:53.557444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:53.558129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:53.558265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:53.558591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] PATH_INFO: `/v3/users/87ac7ab1d3ab4611a8a5729e4019b7ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:53.559652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ac7ab1d3ab4611a8a5729e4019b7ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:53.561573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:53.617075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ac7ab1d3ab4611a8a5729e4019b7ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:53.625871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ac7ab1d3ab4611a8a5729e4019b7ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:53.633889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:53.640382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ac7ab1d3ab4611a8a5729e4019b7ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:53.641541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ac7ab1d3ab4611a8a5729e4019b7ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:53.650617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ac7ab1d3ab4611a8a5729e4019b7ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:53.657030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 727/1449] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:53 2022] DELETE /identity/v3/users/87ac7ab1d3ab4611a8a5729e4019b7ff => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:53.661603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:53.662059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:53.662234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:53.681141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:53.682775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:53.776325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 723/1450] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:53.778057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:53.779549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:53.849004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604c37-6f56-4585-9567-396367dd69e2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:53.864838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:53.868416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:53.869279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:53.869841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:53.883062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:53.885217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:53.905435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 728/1451] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:54.225296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:54.226784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:54.244832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a669d-7216-4217-a94b-0c59ac36d530 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:54.264912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:54.265842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:54.266134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:54.266504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] PATH_INFO: `/v3/projects/5d2a195279184b52875bfee8c9aa5ac6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:54.268413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2a195279184b52875bfee8c9aa5ac6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:54.271156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:54.326873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:54.353208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2a195279184b52875bfee8c9aa5ac6', 'request_id': 'req-2d9052ec-d183-45e5-b6f7-3c68103c9362'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.363652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2a195279184b52875bfee8c9aa5ac6', 'request_id': 'req-2d9052ec-d183-45e5-b6f7-3c68103c9362'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.384699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2a195279184b52875bfee8c9aa5ac6', 'request_id': 'req-2d9052ec-d183-45e5-b6f7-3c68103c9362'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.398022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] The token cache is being invalidate because project 5d2a195279184b52875bfee8c9aa5ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:54.398329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.398980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.400954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 724/1452] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:54 2022] DELETE /identity/v3/projects/5d2a195279184b52875bfee8c9aa5ac6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:54.404842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:54.414494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795ab2-42bf-4b8d-a70d-b762a89169c5 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:54.481463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:54.482293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:54.482733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:54.483030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] PATH_INFO: `/v3/users/da19a9fdbda54991ad980166232fb59f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:54.484408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da19a9fdbda54991ad980166232fb59f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:54.486888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:54.545720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da19a9fdbda54991ad980166232fb59f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.554525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da19a9fdbda54991ad980166232fb59f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.571775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da19a9fdbda54991ad980166232fb59f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.572773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da19a9fdbda54991ad980166232fb59f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.580958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e3d02e-4837-4b6d-a78e-1001c5a41513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da19a9fdbda54991ad980166232fb59f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:54.589512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 729/1453] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:27:54 2022] DELETE /identity/v3/users/da19a9fdbda54991ad980166232fb59f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:55.121886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.123432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:55.137751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9052ec-d183-45e5-b6f7-3c68103c9362 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.198686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.199327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.199601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.199870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] PATH_INFO: `/v3/projects/5304abea04954aea87c5a4225b8c0db6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.200996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5304abea04954aea87c5a4225b8c0db6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.202882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.257176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5304abea04954aea87c5a4225b8c0db6', 'request_id': 'req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:55.264198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5304abea04954aea87c5a4225b8c0db6', 'request_id': 'req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:55.274157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5304abea04954aea87c5a4225b8c0db6', 'request_id': 'req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:55.285387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] The token cache is being invalidate because project 5304abea04954aea87c5a4225b8c0db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:27:55.285673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:55.286311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-c6239c7a-8bb6-4aa9-abb2-9f08ecebe699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:27:55.287818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 725/1454] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:27:55 2022] DELETE /identity/v3/projects/5304abea04954aea87c5a4225b8c0db6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:55.292811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.293253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.293697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.303011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:55.314507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:55.316985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:55.378070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 730/1455] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:55.386201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.386363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.386616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.407473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:55.408780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:55.416885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 726/1456] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:55.421357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.436961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ddc44-7887-43d2-9a62-47cd259be6ef None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.455077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.455774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.456110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.456454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.457192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:27:55.457593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.459305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.471385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 731/1457] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:27:55.476022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.486152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fc8b0-dce5-4389-8e09-2e6d6fd7c7bd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.491022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.491632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.491935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.492192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.492981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.494902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.523384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 727/1458] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:27:55.527659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.537582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24b530-31cf-45d2-936b-a81517fc571c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.598259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.598886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.599163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.599432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.600084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.602062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.626722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 732/1459] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:55.628292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.638950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2740b8-5142-41ab-aee6-2b564ab757bd admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.643926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.644576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.644854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.645126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.645773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.647367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.658367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 728/1460] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:55.662738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.673874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b6c94-e649-4b23-ac34-2cd65dcb4ede admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.679460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.680057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.680363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.680627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.681265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.682870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3de6f-5f93-487a-bac8-fe1ca1098c34 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.693737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 733/1461] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:55.698322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.708899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31398-6b3b-4ec3-8b74-2942f1e674b4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.714411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.715077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.715351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.715640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] PATH_INFO: `/v3/projects/1a8c7fbc09494cec84137633b1ecf8a1/users/4d29d0bda49d4d2fb1dcedfe9bac48b5/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.725756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:55.733310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8c7fbc09494cec84137633b1ecf8a1, user_id=4d29d0bda49d4d2fb1dcedfe9bac48b5, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.736640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:55.753397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 729/1462] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:55 2022] PUT /identity/v3/projects/1a8c7fbc09494cec84137633b1ecf8a1/users/4d29d0bda49d4d2fb1dcedfe9bac48b5/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:27:55.765169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.765517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.765851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.795144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] MFA Rules not processed for user `4d29d0bda49d4d2fb1dcedfe9bac48b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:55.796384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:55.875858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:55.884024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 734/1463] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:55.885249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:55.979009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb05fc-0ce0-45fc-b3c3-46a327079551 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:55.986709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:55.987886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:55.988248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:55.988723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:55.990176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:55.993242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.069986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 730/1464] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:27:55 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:56.071721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.090782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1cfa9-ff25-444c-84dc-0ac9d64d5ed1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.263325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.264105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.264432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.264704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.265647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.268696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.304348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 735/1465] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:27:56 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:27:56.305915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.322244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d081b61-3d29-4a09-9f5e-32553bf2bfe9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.329704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.330696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.331290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.331824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.332875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.335150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.354438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 731/1466] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:56.355696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.367432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27749b6-9e20-4b64-bdc1-72a3ab251e43 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.372610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.373366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.373652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.373928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] PATH_INFO: `/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb/users/2f7282d6eb264bed8a11c3c0118f3662/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.394648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0cda5d07cf426c854ea65be59bf7bb, user_id=2f7282d6eb264bed8a11c3c0118f3662, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.398986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.421094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 736/1467] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:56 2022] PUT /identity/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb/users/2f7282d6eb264bed8a11c3c0118f3662/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:56.425401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.436517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f3fc5-56a2-4f7c-9174-190b1da54785 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.513593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.514756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.515230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.516208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.517078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.519413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefbcfe-d896-43c2-8d70-5fb9eeb01f5c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.537601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 732/1468] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:27:56.538034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.549782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbf511-59cf-4f71-a490-13d7b347ea21 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.554893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.555644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.555924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.556227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] PATH_INFO: `/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb/users/2f7282d6eb264bed8a11c3c0118f3662/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.557744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532a9b1-23c8-4a76-8edb-d1b2161ba6d5 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.558784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532a9b1-23c8-4a76-8edb-d1b2161ba6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.559487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532a9b1-23c8-4a76-8edb-d1b2161ba6d5 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.560459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0cda5d07cf426c854ea65be59bf7bb, user_id=2f7282d6eb264bed8a11c3c0118f3662, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.562399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 733/1469] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:27:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:27:56.563407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e315d6-8683-444d-be20-bf01ce53ab89 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.589103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.589865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.590286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.593488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 737/1470] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:27:56 2022] PUT /identity/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb/users/2f7282d6eb264bed8a11c3c0118f3662/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:27:56.604093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.604792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.604792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.643885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] MFA Rules not processed for user `2f7282d6eb264bed8a11c3c0118f3662`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:56.645392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:56.651317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:27:56.652869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:56.738175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 738/1471] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:27:56.751591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:56.755160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:56.773652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:56.810008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 734/1472] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:27:56.838116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:27:56.858478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edab29-6961-4a13-a491-75b1d1cbd266 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:56.863047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:56.863860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:56.864271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:56.864679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:56.938072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:56.939784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:56.960100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 739/1473] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:27:57.181479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:27:57.183195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:27:57.222110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597787f9-66a7-4935-b59a-173b22f1c78c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:27:57.234257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:27:57.234907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:27:57.235185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:27:57.235454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:27:57.245058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:27:57.246678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:27:57.258333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 735/1474] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:27:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:04.848687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:04.850421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:04.865697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee803b-1671-4b5f-aece-3359ea60214e service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:04.868979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:04.869604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:04.869880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:04.870146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:04.880525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:04.882378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15381dc-0cb9-459a-a1b3-8bc6440717db service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:04.893258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 740/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:04.950268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:04.951841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:05.135458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347157de-b840-45e6-b7d0-a1ba483c9e20 service nova] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:05.140258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:05.141099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:05.141225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:05.141499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] PATH_INFO: `/v3/users/656bc16f0dfa45168d9176fe93394245` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:05.142941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=656bc16f0dfa45168d9176fe93394245)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:05.150738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:05.228184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656bc16f0dfa45168d9176fe93394245'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:05.229259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656bc16f0dfa45168d9176fe93394245'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:05.253861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656bc16f0dfa45168d9176fe93394245'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:05.269810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656bc16f0dfa45168d9176fe93394245'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:05.270572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656bc16f0dfa45168d9176fe93394245'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:05.313817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 736/1476] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:04 2022] DELETE /identity/v3/users/656bc16f0dfa45168d9176fe93394245 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:05.316715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:05.317267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:05.317959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:05.343486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:05.345063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:05.428869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 741/1477] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:05.437009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:05.454229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:05.563144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bceeb-c4d8-4235-916f-9a0344dff1e4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:05.567487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:05.568646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:05.568646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:05.568986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:05.586625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:05.588814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:05.593203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:05.613895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 737/1478] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:05.929111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:05.930747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:05.951761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71ee82-a1e6-48a4-bdbd-31259de30e4e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:05.958307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:05.959452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:05.959844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:05.960217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] PATH_INFO: `/v3/projects/a86c05caf37e459fbbd42bb6f62a6f90` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:05.962526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86c05caf37e459fbbd42bb6f62a6f90)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:05.965970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:06.023085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86c05caf37e459fbbd42bb6f62a6f90', 'request_id': 'req-643924e5-00aa-4559-94dc-30b6481f92e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.040754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86c05caf37e459fbbd42bb6f62a6f90', 'request_id': 'req-643924e5-00aa-4559-94dc-30b6481f92e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.050284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86c05caf37e459fbbd42bb6f62a6f90', 'request_id': 'req-643924e5-00aa-4559-94dc-30b6481f92e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.062657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] The token cache is being invalidate because project a86c05caf37e459fbbd42bb6f62a6f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:06.062952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.064184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.066281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 742/1479] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:05 2022] DELETE /identity/v3/projects/a86c05caf37e459fbbd42bb6f62a6f90 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:06.069672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:06.079562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38800-db35-4340-a4b7-f3ea05e3e77f service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:06.143986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:06.144740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.145056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.145427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] PATH_INFO: `/v3/users/82bd4e33efca4543b11267457cd2db8a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:06.147164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bd4e33efca4543b11267457cd2db8a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:06.151400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:06.204562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bd4e33efca4543b11267457cd2db8a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.206412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bd4e33efca4543b11267457cd2db8a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.219910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bd4e33efca4543b11267457cd2db8a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.231022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bd4e33efca4543b11267457cd2db8a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.231699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bf27387e-3fa0-4878-9462-ce9bea7446d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bd4e33efca4543b11267457cd2db8a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.245133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 738/1480] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:06 2022] DELETE /identity/v3/users/82bd4e33efca4543b11267457cd2db8a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:06.538144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:06.540204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:06.558620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:06.567948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643924e5-00aa-4559-94dc-30b6481f92e8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:06.635678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:06.636400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.636678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.636940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] PATH_INFO: `/v3/projects/0dc92d82c0084f94b442c4bf559dc843` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:06.638185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc92d82c0084f94b442c4bf559dc843)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:06.640086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:06.701778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc92d82c0084f94b442c4bf559dc843', 'request_id': 'req-155b7009-d8b8-4b3d-b5a5-cda064a6367f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.709315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc92d82c0084f94b442c4bf559dc843', 'request_id': 'req-155b7009-d8b8-4b3d-b5a5-cda064a6367f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.719633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc92d82c0084f94b442c4bf559dc843', 'request_id': 'req-155b7009-d8b8-4b3d-b5a5-cda064a6367f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.746744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] The token cache is being invalidate because project 0dc92d82c0084f94b442c4bf559dc843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:06.747364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.750936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-155b7009-d8b8-4b3d-b5a5-cda064a6367f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:06.752554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 743/1481] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:06 2022] DELETE /identity/v3/projects/0dc92d82c0084f94b442c4bf559dc843 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:06.757591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.757881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.758223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:06.783075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:06.784488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:06.862299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 739/1482] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:06.873887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.874426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.874834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:06.899961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:06.901299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:06.914194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 744/1483] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:06.918467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:06.932758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8429e0-a8cc-45da-9d7e-99064982cb5d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:06.937497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:06.938696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.938696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.938873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:06.939518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:06.939817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:06.941951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:06.965746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 740/1484] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:06.977387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:06.991995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece907b4-aebf-4ba0-9a19-b1df8cffc516 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:06.998488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:06.999338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:06.999596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:06.999913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.000527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.002427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.040276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 745/1485] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:06 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:07.044579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.057287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66ed49-f330-4975-9995-3eda66ff58d2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.058651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:07.147988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.147988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.147988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.147988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.149734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.149806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.171988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.174438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.181359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 741/1486] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:07.182724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.194482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c18e0-04fd-4196-a09e-ee9a4943601e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.197333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce45742-5979-445f-80d1-96ed3ad88a2d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.199715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.200608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.200868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.202019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] PATH_INFO: `/v3/users/7c6bdec02848467c9ccb30965b19b63f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.203359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6bdec02848467c9ccb30965b19b63f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.205915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.206660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.207000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.207900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.208410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.209000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.211299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.231209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 742/1487] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:07.232815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.244627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517874-3066-4584-a6b8-aec2c5ae7409 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.250085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.251006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.251312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.251604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.252403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.254324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.270348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 743/1488] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:07.271382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.281621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bec553-aaf0-4b98-9944-4f206c5c6f9d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.282952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6bdec02848467c9ccb30965b19b63f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:07.286682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.287390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.287682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.287978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] PATH_INFO: `/v3/projects/e477a14e0acd4c0fb5747a4624b8b07c/users/2ddb74b6655d4b7796d6cf04d494012a/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.291132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6bdec02848467c9ccb30965b19b63f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:07.293950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:07.306808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e477a14e0acd4c0fb5747a4624b8b07c, user_id=2ddb74b6655d4b7796d6cf04d494012a, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.309596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702ea9-a9ae-484f-b84e-0432b16061e2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.309933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6bdec02848467c9ccb30965b19b63f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:07.312449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6bdec02848467c9ccb30965b19b63f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:07.326767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 744/1489] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:07 2022] PUT /identity/v3/projects/e477a14e0acd4c0fb5747a4624b8b07c/users/2ddb74b6655d4b7796d6cf04d494012a/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:07.339808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d740a073-d922-4b1b-acca-1f3a758647aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6bdec02848467c9ccb30965b19b63f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:07.346076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.346460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.347851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.348612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 746/1490] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:07 2022] DELETE /identity/v3/users/7c6bdec02848467c9ccb30965b19b63f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:07.352860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.353291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.353774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.373506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:07.375095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.377181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] MFA Rules not processed for user `2ddb74b6655d4b7796d6cf04d494012a`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:07.378445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.433859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 747/1491] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:07.442691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 745/1492] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:07.443641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.445002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.446882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.448412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.462463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab2865-fb85-45eb-8fb9-cb4d89b977a4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.467039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.467690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.467993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.468325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.468966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.470859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.498511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 746/1493] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:07.502461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.505095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d76f7-001e-4a89-bc8a-a304089c2415 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.520245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:07.555367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.555986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.556298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.556567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.566808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba2bf-4a15-46b6-932c-ac41350814f5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.571712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.572356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.572633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.573282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.573534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.575424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.603712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:07.604529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 747/1494] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:07.608697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.619086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ea664-f5c2-472a-b8ae-1da1f8608fb1 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.621337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.622767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.623791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.624436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.624720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.624993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.625642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.627239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.632019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 748/1495] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:28:07.637618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 748/1496] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:07.641542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.651093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f40f-140a-4b58-8136-05eb6406ebeb service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.655524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.656235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.656516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.656781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] PATH_INFO: `/v3/projects/5b13cb5ccb5248d0af86d0a48e827643/users/cd3fcf9a660d47909e9c85a22c2f9f91/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.675099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b13cb5ccb5248d0af86d0a48e827643, user_id=cd3fcf9a660d47909e9c85a22c2f9f91, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.678025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.693988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 749/1497] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:07 2022] PUT /identity/v3/projects/5b13cb5ccb5248d0af86d0a48e827643/users/cd3fcf9a660d47909e9c85a22c2f9f91/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:07.698242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.708759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e9c3-2edb-4531-959a-42d06d489423 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.773530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.774199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.774530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.774811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.775531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.777348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b6e07-0bd4-44a8-b3f3-a337f361bc66 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.788610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 749/1498] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:07.793339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:07.805502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c141c2-7c58-4a12-9f62-b233a13c0dc8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:07.810192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:07.810957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.811239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.811509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] PATH_INFO: `/v3/projects/5b13cb5ccb5248d0af86d0a48e827643/users/cd3fcf9a660d47909e9c85a22c2f9f91/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.813424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b13cb5ccb5248d0af86d0a48e827643, user_id=cd3fcf9a660d47909e9c85a22c2f9f91, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:07.816232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:07.827392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 750/1499] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:07 2022] PUT /identity/v3/projects/5b13cb5ccb5248d0af86d0a48e827643/users/cd3fcf9a660d47909e9c85a22c2f9f91/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:07.837860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:07.838150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:07.838429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:07.869283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] MFA Rules not processed for user `cd3fcf9a660d47909e9c85a22c2f9f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:07.870878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:07.979038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 750/1500] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:07.999865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:08.001609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:08.038426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:08.045579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:08.103061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:08.146472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf311f3-1907-4cb3-bf51-0bf5e2262c3e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:08.158688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:08.159550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.159832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.160443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.172719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:08.174441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:08.177167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f88120-f5ef-46b7-8c73-b3896643dace None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:08.182262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:08.183124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.183707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.184051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] PATH_INFO: `/v3/projects/12f28bd3acf348b1923cedbbc6e47edd` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.185325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f28bd3acf348b1923cedbbc6e47edd)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:08.187300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:08.191659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 751/1501] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:28:08.243534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f28bd3acf348b1923cedbbc6e47edd', 'request_id': 'req-e0964424-b3b9-4cee-a6f4-99985026d1ce'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.250606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f28bd3acf348b1923cedbbc6e47edd', 'request_id': 'req-e0964424-b3b9-4cee-a6f4-99985026d1ce'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.259359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f28bd3acf348b1923cedbbc6e47edd', 'request_id': 'req-e0964424-b3b9-4cee-a6f4-99985026d1ce'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.271638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] The token cache is being invalidate because project 12f28bd3acf348b1923cedbbc6e47edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:08.271922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.272628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.274314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 751/1502] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:08 2022] DELETE /identity/v3/projects/12f28bd3acf348b1923cedbbc6e47edd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:08.278347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:08.291030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd74489-0fb4-419f-98f4-dc0274ef5a69 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:08.370161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:08.372597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.372941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.373165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] PATH_INFO: `/v3/users/39d6b1fd0101427d8c038a6a64d0af7c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.376355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d6b1fd0101427d8c038a6a64d0af7c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:08.396183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:08.464376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d6b1fd0101427d8c038a6a64d0af7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.466793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:08.474502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d6b1fd0101427d8c038a6a64d0af7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.489271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d6b1fd0101427d8c038a6a64d0af7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.490003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d6b1fd0101427d8c038a6a64d0af7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.500581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d6b1fd0101427d8c038a6a64d0af7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.508209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 752/1503] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:08 2022] DELETE /identity/v3/users/39d6b1fd0101427d8c038a6a64d0af7c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:08.643776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:08.645348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:08.661687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964424-b3b9-4cee-a6f4-99985026d1ce admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:08.870014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:08.870673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.870947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.871212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] PATH_INFO: `/v3/users/2ddb74b6655d4b7796d6cf04d494012a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.872515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ddb74b6655d4b7796d6cf04d494012a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:08.872837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:08.874314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:08.874494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:08.893710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c80cb-bf92-46dc-ae29-d94aac44fe06 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:08.898591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:08.899309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.899585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.899856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] PATH_INFO: `/v3/projects/59014cac09284fd4ac87ef28777787f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.901133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59014cac09284fd4ac87ef28777787f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:08.903755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:08.932245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ddb74b6655d4b7796d6cf04d494012a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.932902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ddb74b6655d4b7796d6cf04d494012a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.934202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:08.949914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ddb74b6655d4b7796d6cf04d494012a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.958238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ddb74b6655d4b7796d6cf04d494012a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.958800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59014cac09284fd4ac87ef28777787f2', 'request_id': 'req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.959109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-980696ff-4b85-43db-95c8-d89fea44662e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ddb74b6655d4b7796d6cf04d494012a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.966342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59014cac09284fd4ac87ef28777787f2', 'request_id': 'req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.972759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 752/1504] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:08 2022] DELETE /identity/v3/users/2ddb74b6655d4b7796d6cf04d494012a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:08.976470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59014cac09284fd4ac87ef28777787f2', 'request_id': 'req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.977721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:08.978243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:08.978516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:08.991969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] The token cache is being invalidate because project 59014cac09284fd4ac87ef28777787f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:08.992651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.993470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7bc7ad70-bdfc-48ce-a2a9-ed843d9bd7e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:08.995710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 753/1505] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:08 2022] DELETE /identity/v3/projects/59014cac09284fd4ac87ef28777787f2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:09.000792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e06fd0-742b-4fab-b29e-bb664d3dea1c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.000792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e06fd0-742b-4fab-b29e-bb664d3dea1c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.001522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e06fd0-742b-4fab-b29e-bb664d3dea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.002716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:09.004079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.031766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-02e06fd0-742b-4fab-b29e-bb664d3dea1c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:09.032948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02e06fd0-742b-4fab-b29e-bb664d3dea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.079643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 753/1506] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:09.089877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.091689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.148426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 754/1507] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:09.151919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.151919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.151919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.182653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:09.186688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.196478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5e70-321b-4dc3-8ba2-99112e1a903b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.200375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.203586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.203586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.203586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.204074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 755/1508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:09.205158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.215156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.216669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.240432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbe92e-bbea-436a-90a4-6ea7d6bf96e2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.245274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 754/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:09.247698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.248840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.249275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.249687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.250671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:09.251357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.253337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.285437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 756/1510] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:09.286086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.297519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60592bef-6c17-42a8-b586-b6089e1ddd56 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.305218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.305940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.306380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.306648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.307281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.309362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.377327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 755/1511] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:09 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:09.382018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.395437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113dd8b-2b16-4698-a04d-e5635b96255a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.462917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:09.545851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.546489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.546763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.547031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.547684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.549637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.593857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 757/1512] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:09 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:09.599245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.613792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c72ba-85e7-461e-aa39-4458d7ae41d1 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.619614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.621417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.621870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.622236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.623507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.625461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.644962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 756/1513] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:09.645170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.658620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7ebf-4fcd-48a1-a10a-a9ee46bc1287 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.664842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.665944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.666504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.667045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.667955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.670223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.681565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.684102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.709043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0db5a-467a-4e54-a9f1-706667d5bce5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.716213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 758/1514] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:09.717125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.719431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.720453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.720992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.722321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] PATH_INFO: `/v3/projects/e477a14e0acd4c0fb5747a4624b8b07c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.723761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e477a14e0acd4c0fb5747a4624b8b07c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.725774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.728314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f708-93bc-4dab-93be-c21e6eaf87bc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.733449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.734154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.734428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.734697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] PATH_INFO: `/v3/projects/c75d44185d404651872add96b9b6df16/users/497cd24a23e34384beed36bd1a21f75c/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.751962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75d44185d404651872add96b9b6df16, user_id=497cd24a23e34384beed36bd1a21f75c, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.754488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed3676-aa0b-4fc6-bae8-a2bb0e78c76e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.765248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:09.770506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 759/1515] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:09 2022] PUT /identity/v3/projects/c75d44185d404651872add96b9b6df16/users/497cd24a23e34384beed36bd1a21f75c/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:09.780828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.781157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.781408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.792687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e477a14e0acd4c0fb5747a4624b8b07c', 'request_id': 'req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:09.800005 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e477a14e0acd4c0fb5747a4624b8b07c', 'request_id': 'req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:09.810298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] MFA Rules not processed for user `497cd24a23e34384beed36bd1a21f75c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:09.811278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e477a14e0acd4c0fb5747a4624b8b07c', 'request_id': 'req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:09.811541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.825447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] The token cache is being invalidate because project e477a14e0acd4c0fb5747a4624b8b07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:09.825826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:09.826749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:09.828794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 757/1516] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:09 2022] DELETE /identity/v3/projects/e477a14e0acd4c0fb5747a4624b8b07c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:09.833024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.837259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.896783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 760/1517] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:09.901003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:09.902387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:09.926133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bde6c3-0abf-4ebf-9bbe-6044f69b0186 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.931369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.932074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.932389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.932663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] PATH_INFO: `/v3/users/cd3fcf9a660d47909e9c85a22c2f9f91` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.933972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd3fcf9a660d47909e9c85a22c2f9f91)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.937057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:09.976313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a97d1-56aa-4d4b-ad56-8461f73cee37 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:09.981060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:09.981585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:09.981859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:09.982123 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:09.982764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:09.984775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.005441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd3fcf9a660d47909e9c85a22c2f9f91'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.006306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd3fcf9a660d47909e9c85a22c2f9f91'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.010596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:10.022156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 761/1518] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:09 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:10.023100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.024858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.027665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd3fcf9a660d47909e9c85a22c2f9f91'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.037098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd3fcf9a660d47909e9c85a22c2f9f91'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.037893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd3fcf9a660d47909e9c85a22c2f9f91'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.051477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 758/1519] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:09 2022] DELETE /identity/v3/users/cd3fcf9a660d47909e9c85a22c2f9f91 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:10.092700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e84f-daf8-47fd-bd41-86961af2fc66 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.158144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.158965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.159308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.159572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.160349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.162452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.187601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 762/1520] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:10 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:10.191680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.203851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400585-1e4e-48a5-bacb-b3510d782730 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.228225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.228225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.228225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.228225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.229470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.229470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.252101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 759/1521] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:10.256855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.278355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc5a1-5e61-4b2b-815d-7a7c8dda3dee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.289465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.290934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.291220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.291501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] PATH_INFO: `/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1/users/68a4a98865ec44149d207643c01e2678/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.315067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d83ccd995ef4b5aa89cc9c20a4cb3a1, user_id=68a4a98865ec44149d207643c01e2678, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.318171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.335980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 763/1522] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:10 2022] PUT /identity/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1/users/68a4a98865ec44149d207643c01e2678/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:10.340080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.350531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa7811-7268-48f3-aa11-9651ad889f0b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.385285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:10.407595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.410024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.432763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.432935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.433377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.433660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.434421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.436282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.458207 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 760/1523] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:10.463950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.477617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322575ea-6bd1-478b-b0a5-6408ba4c7ee4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.482439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.483177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.483527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.483881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] PATH_INFO: `/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1/users/68a4a98865ec44149d207643c01e2678/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.485882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d83ccd995ef4b5aa89cc9c20a4cb3a1, user_id=68a4a98865ec44149d207643c01e2678, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.488674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.504092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 761/1524] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:10 2022] PUT /identity/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1/users/68a4a98865ec44149d207643c01e2678/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:10.505103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.512771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ca80-c900-48de-8b8e-ad2f5929af57 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.518013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.537276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:10.595921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.596629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.596957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.597233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] PATH_INFO: `/v3/projects/5b13cb5ccb5248d0af86d0a48e827643` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.598373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b13cb5ccb5248d0af86d0a48e827643)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.600753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.605550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a85f-1af8-41c4-986e-1f2236773cef admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.611887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.612838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.613379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.613982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] PATH_INFO: `/v3/users/4d29d0bda49d4d2fb1dcedfe9bac48b5` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.615490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d29d0bda49d4d2fb1dcedfe9bac48b5)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.617846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.676713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b13cb5ccb5248d0af86d0a48e827643', 'request_id': 'req-18101611-51b9-44d1-a073-0dfb7dfbe4dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.692759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b13cb5ccb5248d0af86d0a48e827643', 'request_id': 'req-18101611-51b9-44d1-a073-0dfb7dfbe4dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.701722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d29d0bda49d4d2fb1dcedfe9bac48b5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.703004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d29d0bda49d4d2fb1dcedfe9bac48b5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.708974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b13cb5ccb5248d0af86d0a48e827643', 'request_id': 'req-18101611-51b9-44d1-a073-0dfb7dfbe4dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.726142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d29d0bda49d4d2fb1dcedfe9bac48b5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.727564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] The token cache is being invalidate because project 5b13cb5ccb5248d0af86d0a48e827643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:10.736526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.739452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-18101611-51b9-44d1-a073-0dfb7dfbe4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.740213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d29d0bda49d4d2fb1dcedfe9bac48b5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.741704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b021ff6f-cf5c-45e1-a7cb-56b1f363b9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d29d0bda49d4d2fb1dcedfe9bac48b5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:10.751758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 764/1525] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:10 2022] DELETE /identity/v3/projects/5b13cb5ccb5248d0af86d0a48e827643 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:10.756841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334a7b-efd2-456b-97b8-f864e8acc3e0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.757137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334a7b-efd2-456b-97b8-f864e8acc3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.757417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334a7b-efd2-456b-97b8-f864e8acc3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.758609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 762/1526] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:10 2022] DELETE /identity/v3/users/4d29d0bda49d4d2fb1dcedfe9bac48b5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:10.759470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83a86-9342-4320-98e1-35bfc1e07f05 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.759758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83a86-9342-4320-98e1-35bfc1e07f05 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.760032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83a86-9342-4320-98e1-35bfc1e07f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.782211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6eb83a86-9342-4320-98e1-35bfc1e07f05 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:10.783481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6eb83a86-9342-4320-98e1-35bfc1e07f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.792408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ca334a7b-efd2-456b-97b8-f864e8acc3e0 None None] MFA Rules not processed for user `68a4a98865ec44149d207643c01e2678`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:10.794357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ca334a7b-efd2-456b-97b8-f864e8acc3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.809271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:10.853607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 763/1527] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:10.861128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.861413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.861725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.884011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 765/1528] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:10.884396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:10.884499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.884499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.884499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.884704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.891635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 764/1529] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:10.892891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.894199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.905357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:10.907542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.922558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 766/1530] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:10.923581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:10.924966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:10.959061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:10.970978 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f3c0-4e1e-4da3-9868-05dd5d626726 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.975004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.975660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.975934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.976231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:10.988136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:10.989750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:10.992533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643b7cf-fdfd-4f5b-9179-062812915ee3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:10.996476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:10.997358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:10.998059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:10.998522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.001774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 765/1531] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:11.002632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.012105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.014260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.024940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5b25-e506-4d34-88ef-a1b33a9fbde8 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.028937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 767/1532] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:11.043923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.044633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.044915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.045184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.045851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:11.046160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.048278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.092795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 766/1533] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:11.097398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.124302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7855fe-a277-4785-a907-dc42b684fa27 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.146400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.146400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.146400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.146400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.146722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.152393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.217405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 768/1534] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:11 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:11.218863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.219769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.221735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:11.244761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f490e-4c43-4ba5-abf5-10bec707f9e2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.364170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab46274-a955-4e82-8795-588ae85fd6d9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.368326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.368992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.369271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.369539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] PATH_INFO: `/v3/users/497cd24a23e34384beed36bd1a21f75c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.370836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497cd24a23e34384beed36bd1a21f75c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.373907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.375026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.375637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.376096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.376566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.377038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.379797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.415875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 769/1535] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:11 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:11.420197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.432728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef24c27-93bc-4363-8d3b-615723909d0b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.439781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.440574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.440903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.441913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.442049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.443998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.478867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 770/1536] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:11.482808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.503704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497cd24a23e34384beed36bd1a21f75c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:11.506310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497cd24a23e34384beed36bd1a21f75c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:11.509573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a3269-5475-4a42-a351-557c2dbb84e1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.519994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:11.522675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.526376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.526668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.528363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.529139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.536597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.539105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497cd24a23e34384beed36bd1a21f75c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:11.556487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497cd24a23e34384beed36bd1a21f75c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:11.557264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-852f263e-ee55-433b-835e-4002263cf576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497cd24a23e34384beed36bd1a21f75c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:11.574111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 771/1537] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:11.578531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.582878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:11.591630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 767/1538] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:11 2022] DELETE /identity/v3/users/497cd24a23e34384beed36bd1a21f75c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:11.596751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.596751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.596751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.615806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4801d-c1dd-4af3-8aaf-0ce51cdc2f83 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.632368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:11.634168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:11.709376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 768/1539] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:11.719517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.720246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.720525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.720796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] PATH_INFO: `/v3/projects/116b194c617a44dba998c5e6a053412f/users/355afc838ded491e8d1ea8d217c36daf/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.735279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.737338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:11.750587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116b194c617a44dba998c5e6a053412f, user_id=355afc838ded491e8d1ea8d217c36daf, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:11.753964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6a93-bb1c-4949-96d1-1fcf85343850 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:11.767903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:11.803430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 772/1540] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:11 2022] PUT /identity/v3/projects/116b194c617a44dba998c5e6a053412f/users/355afc838ded491e8d1ea8d217c36daf/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:11.814606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.815196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.830199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648c18-282b-4eb0-899e-acfdc0380d71 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:11.831270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.880192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] MFA Rules not processed for user `355afc838ded491e8d1ea8d217c36daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:11.882876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:11.912840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:11.913774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:11.913884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:11.914166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:11.986372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:11.995717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 773/1541] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:11.996758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:11.998178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:12.016921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.018960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.031903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 769/1542] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:12.033151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.045376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7cc37-e726-4379-975c-c37188394ff9 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.051376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.052140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.052519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.052855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.053871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.055890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.090956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 770/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:12 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:12.095013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.119296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418a039-82da-4f23-9a91-e3fdd208c43b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.260301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.261007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.261520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.261843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] PATH_INFO: `/v3/projects/1a8c7fbc09494cec84137633b1ecf8a1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.269280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8c7fbc09494cec84137633b1ecf8a1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.272658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.313638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ff7dc-5414-48a0-aa38-2e772cc13fe2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.327139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.331835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.332310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.332685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.334371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.341120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.369675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:12.382216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8c7fbc09494cec84137633b1ecf8a1', 'request_id': 'req-436c2fee-4d11-4793-8a2d-d999f0cc26f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.391064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8c7fbc09494cec84137633b1ecf8a1', 'request_id': 'req-436c2fee-4d11-4793-8a2d-d999f0cc26f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.395077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 771/1544] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:12 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:12.395975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.409929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8c7fbc09494cec84137633b1ecf8a1', 'request_id': 'req-436c2fee-4d11-4793-8a2d-d999f0cc26f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.414354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97104-2812-4892-8cb2-09a59eef2bc5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.426189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] The token cache is being invalidate because project 1a8c7fbc09494cec84137633b1ecf8a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:12.429375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.430289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.433492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 774/1545] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:11 2022] DELETE /identity/v3/projects/1a8c7fbc09494cec84137633b1ecf8a1 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:12.437332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.439822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:12.489837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.490938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.490938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.491155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.491874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.493658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.506242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 772/1546] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:12.511567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.513478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:12.542576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614f237-01c2-4a66-9300-5c2ce46ef427 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.555100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.556167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.557240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.557755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] PATH_INFO: `/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8/users/1972f975d50140168007d267abd31393/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.566211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c2fee-4d11-4793-8a2d-d999f0cc26f1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.577785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d1b7b08197445fbeadb3f63c5b54e8, user_id=1972f975d50140168007d267abd31393, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.581055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.588182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.589398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.590208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.591200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] PATH_INFO: `/v3/users/2f7282d6eb264bed8a11c3c0118f3662` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.593382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7282d6eb264bed8a11c3c0118f3662)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.596512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.603073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 773/1547] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:12 2022] PUT /identity/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8/users/1972f975d50140168007d267abd31393/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:12.609440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.647898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:12.657653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7282d6eb264bed8a11c3c0118f3662'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.666965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7282d6eb264bed8a11c3c0118f3662'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.681451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7282d6eb264bed8a11c3c0118f3662'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.682285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7282d6eb264bed8a11c3c0118f3662'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.691472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7282d6eb264bed8a11c3c0118f3662'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.696551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c29b-578a-4a58-a3f8-582c926b6a8b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.698062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 775/1548] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:12 2022] DELETE /identity/v3/users/2f7282d6eb264bed8a11c3c0118f3662 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:12.700272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.700473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:12.766555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:12.792929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf02b1d-c23e-45c5-95fc-767c49617f1d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.799272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.800002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.800510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.800907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] PATH_INFO: `/v3/projects/c75d44185d404651872add96b9b6df16` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.802030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75d44185d404651872add96b9b6df16)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.803045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.803654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.803947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.804060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.804321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.805505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.807195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.835142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 774/1549] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:12.837085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.839625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:12.861806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e6122-b07d-4592-805a-f38325ac274f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:12.877302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:12.879431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.880558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.881170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] PATH_INFO: `/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8/users/1972f975d50140168007d267abd31393/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.886576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d1b7b08197445fbeadb3f63c5b54e8, user_id=1972f975d50140168007d267abd31393, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:12.890210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75d44185d404651872add96b9b6df16', 'request_id': 'req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.897279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bfd7f-8055-4382-b0d7-b954b46d8edd admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:12.911604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 775/1550] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:12 2022] PUT /identity/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8/users/1972f975d50140168007d267abd31393/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:12.913726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75d44185d404651872add96b9b6df16', 'request_id': 'req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.922344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:12.923608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:12.923891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:12.926350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75d44185d404651872add96b9b6df16', 'request_id': 'req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.954972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] The token cache is being invalidate because project c75d44185d404651872add96b9b6df16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:12.955331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.956014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:12.957691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 776/1551] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:12 2022] DELETE /identity/v3/projects/c75d44185d404651872add96b9b6df16 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:12.962450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:12.968229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.016343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] MFA Rules not processed for user `1972f975d50140168007d267abd31393`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:13.019425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.158338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c21c2b-6da0-49ed-ac20-6f4565832ffc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:13.222415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 776/1552] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:13.222415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:13.225494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.274062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:13.274648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.274928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.275211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] PATH_INFO: `/v3/users/68a4a98865ec44149d207643c01e2678` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.277531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a4a98865ec44149d207643c01e2678)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:13.280594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:13.295174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:13.339245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06c26-1da0-4e69-9e0b-f8661e43fe6e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:13.349248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:13.350466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:13.352222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.354452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.354739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] PATH_INFO: `/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.358258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0cda5d07cf426c854ea65be59bf7bb)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:13.363364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:13.394851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a4a98865ec44149d207643c01e2678'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.405411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a4a98865ec44149d207643c01e2678'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.432787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a4a98865ec44149d207643c01e2678'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.434418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a4a98865ec44149d207643c01e2678'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.448933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a4a98865ec44149d207643c01e2678'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.468141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 777/1553] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:12 2022] DELETE /identity/v3/users/68a4a98865ec44149d207643c01e2678 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:13.468141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:13.468141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.468667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0cda5d07cf426c854ea65be59bf7bb', 'request_id': 'req-087d640b-a2ad-410b-a630-04e0fad7b1c2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.468667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0cda5d07cf426c854ea65be59bf7bb', 'request_id': 'req-087d640b-a2ad-410b-a630-04e0fad7b1c2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.481753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0cda5d07cf426c854ea65be59bf7bb', 'request_id': 'req-087d640b-a2ad-410b-a630-04e0fad7b1c2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.497721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] The token cache is being invalidate because project 7c0cda5d07cf426c854ea65be59bf7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:13.498380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.499455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-087d640b-a2ad-410b-a630-04e0fad7b1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:13.521986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 777/1554] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:13 2022] DELETE /identity/v3/projects/7c0cda5d07cf426c854ea65be59bf7bb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:13.523667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.524481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.525034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.548602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a1254-e4d7-40c8-8071-95cfd24f277b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:13.551953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:13.552605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.552878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.553147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.553514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:13.554814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.555026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.645482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:13.648472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2828ea-62c2-48b2-8646-56133ec91e7d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:13.676546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 778/1555] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:13.677976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 778/1556] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:13.691851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.692800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.693285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.739773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:13.745319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:13.770072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 779/1557] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:13.774253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:13.789965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88e3a-56df-42be-a3cb-550d60823b0c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:13.808157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:13.808807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.809110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.809455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.810120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:13.810422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:13.815740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:13.836835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 779/1558] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:13.842694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:13.855779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4ceb6-8cd7-4db7-a43a-19f99ccdd1a4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:13.860709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:13.861763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:13.862153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:13.862425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:13.863060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:13.865388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:13.911316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:13.922918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 780/1559] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:13 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:13.929960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85b27e-3beb-4564-9b7c-0ef7ce79bea6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.007949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:14.029917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.030654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.030877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.031171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.031932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.034308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.060708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 780/1560] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:13 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:14.065043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.079819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbf80-7896-4973-a312-8f30d0a44e3d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.085068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.085781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.086196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.086534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.087257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.089224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.106423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.107200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 781/1561] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:14.119025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.122068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.123061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae9876-bfac-435d-9fdc-59e869fd8de2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.130607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.131668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.132012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.132385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.133139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.134891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.152602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a526b-a36f-4500-a6f7-aa3272ab2027 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.153896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 781/1562] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:14.154945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.168966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54580b2-b948-4b2a-bc04-c1ba325c7307 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.169447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.176003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.176813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.177179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.177521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] PATH_INFO: `/v3/projects/951180f93a5e4676b5294930b163a71e/users/998b037068874210a1e13b98e0d8ad5b/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.186303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.186591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.186865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] PATH_INFO: `/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.191777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d83ccd995ef4b5aa89cc9c20a4cb3a1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.199943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.201427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=951180f93a5e4676b5294930b163a71e, user_id=998b037068874210a1e13b98e0d8ad5b, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.206242 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b5a0f-f4af-473d-a0a7-dea8fcf6f77a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.231839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:14.236194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 782/1563] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:14 2022] PUT /identity/v3/projects/951180f93a5e4676b5294930b163a71e/users/998b037068874210a1e13b98e0d8ad5b/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:14.256882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4ba6-4caa-4c24-aeb5-ec17cc4eec62 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.258855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4ba6-4caa-4c24-aeb5-ec17cc4eec62 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.259149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4ba6-4caa-4c24-aeb5-ec17cc4eec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.317943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-951e4ba6-4caa-4c24-aeb5-ec17cc4eec62 None None] MFA Rules not processed for user `998b037068874210a1e13b98e0d8ad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:14.319510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-951e4ba6-4caa-4c24-aeb5-ec17cc4eec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.339868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d83ccd995ef4b5aa89cc9c20a4cb3a1', 'request_id': 'req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.350414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d83ccd995ef4b5aa89cc9c20a4cb3a1', 'request_id': 'req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.365626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d83ccd995ef4b5aa89cc9c20a4cb3a1', 'request_id': 'req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.386541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] The token cache is being invalidate because project 6d83ccd995ef4b5aa89cc9c20a4cb3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:14.386832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.387790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.389892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 782/1564] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:14 2022] DELETE /identity/v3/projects/6d83ccd995ef4b5aa89cc9c20a4cb3a1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:14.390743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.399217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.412857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 783/1565] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:14.412857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.412857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.412857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.431591 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:14.432862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.485473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5395f4d-8ba6-4be6-affe-5c16d93fa1cd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.494055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.494862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.495276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.495539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] PATH_INFO: `/v3/users/355afc838ded491e8d1ea8d217c36daf` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.498155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=355afc838ded491e8d1ea8d217c36daf)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.502091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.514767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 784/1566] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:14.515717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.517415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.561224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355afc838ded491e8d1ea8d217c36daf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.569674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355afc838ded491e8d1ea8d217c36daf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.581201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:14.594068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355afc838ded491e8d1ea8d217c36daf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.594913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355afc838ded491e8d1ea8d217c36daf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.608665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f6f69288-686e-4aac-a725-91aaa91e8dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355afc838ded491e8d1ea8d217c36daf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:14.616714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 783/1567] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:14 2022] DELETE /identity/v3/users/355afc838ded491e8d1ea8d217c36daf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:14.618199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe599d-52e7-4752-b6b0-1ecd867246a7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.618485 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe599d-52e7-4752-b6b0-1ecd867246a7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.618760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe599d-52e7-4752-b6b0-1ecd867246a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.631333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46bba4-4e79-43e4-b6f2-f841d170c56d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.639554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dabe599d-52e7-4752-b6b0-1ecd867246a7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:14.640820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dabe599d-52e7-4752-b6b0-1ecd867246a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.654993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:14.690308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.690972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.691252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.691654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.693626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.727319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 784/1568] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:14.745899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.746203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.746596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.771092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.773175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:14.773302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.774537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.787015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 785/1569] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:14.788175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.802185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f9a1-602b-45cd-9f51-94ce680dff77 service glance] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.802315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 785/1570] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:14.809416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.809416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.809416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.809416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.812742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:14.813205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.814715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.814999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.816232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:14.829739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 786/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:14.834130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.845675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b9cd-c590-417f-bdd6-81d1431018b6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.851796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.852624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.853030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.853567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:14.854311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:14.856855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:14.897215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 787/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:14.898684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0976d7e-84dd-4cb0-b4f0-8f1aacf29088 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:14.901755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:14.964628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:14.965466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:14.965860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:14.966313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.044814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:15.061120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28cb3-72b7-41f9-87c4-524288d416fb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.067584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.068073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.068820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.069133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.069471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.070234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.071571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.072349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.083029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 786/1573] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:28:15.101882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 788/1574] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:14 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:15.106261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.120186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d860c5-1c67-4c1a-b930-9f2cf1a9004e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.135874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.136764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.137007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.137334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.137989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.141070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.156905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 787/1575] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:15.160247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.175625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de0cd5-fcf1-4cb5-a262-8cf01f8533a1 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.188642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.191911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.192343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.193196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.194355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.198208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02256772-f9d5-4f53-8c51-99e32598e514 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.223129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 789/1576] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:15.227577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.238581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:15.245959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a1125-e9ca-4c6d-a770-9d79feb07dfe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.252736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.254620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.255160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.255730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] PATH_INFO: `/v3/projects/aab762db6e654d0b8e4ed73dd80117e6/users/13b7bec530af43caa7a00908c89ca482/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.286068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab762db6e654d0b8e4ed73dd80117e6, user_id=13b7bec530af43caa7a00908c89ca482, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.289492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.320017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 788/1577] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:15 2022] PUT /identity/v3/projects/aab762db6e654d0b8e4ed73dd80117e6/users/13b7bec530af43caa7a00908c89ca482/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:15.322942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.323409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.323820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.380111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] MFA Rules not processed for user `13b7bec530af43caa7a00908c89ca482`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:15.381411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:15.426958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.428815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:15.475073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 790/1578] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:15.487911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.494343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:15.561247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040fe41-15f5-4eb1-8e80-1e5cf6dc8586 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.579872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.581800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.582175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.582844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] PATH_INFO: `/v3/projects/116b194c617a44dba998c5e6a053412f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.585713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=116b194c617a44dba998c5e6a053412f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.592527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.633142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97789baa-105f-4fca-82e8-af04a2fbc270 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.638561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.639420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.639699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.639964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.651791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.653248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.657263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116b194c617a44dba998c5e6a053412f', 'request_id': 'req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.666179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116b194c617a44dba998c5e6a053412f', 'request_id': 'req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.671715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 791/1579] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:15.686424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116b194c617a44dba998c5e6a053412f', 'request_id': 'req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.693741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.705885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1d05d-7902-4a3c-9f51-b3ed7fad3e58 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.706622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] The token cache is being invalidate because project 116b194c617a44dba998c5e6a053412f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:15.706921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.707758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.709778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 789/1580] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:15 2022] DELETE /identity/v3/projects/116b194c617a44dba998c5e6a053412f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:15.713739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:15.715269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:15.771204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.772325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.772788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.773236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] PATH_INFO: `/v3/users/13b7bec530af43caa7a00908c89ca482` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.774901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b7bec530af43caa7a00908c89ca482)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.777524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.780467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:15.805488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5fae-35c3-4480-8b5d-1b2a297ecc7c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:15.812878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:15.814860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.815335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.815756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] PATH_INFO: `/v3/users/1972f975d50140168007d267abd31393` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.818171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1972f975d50140168007d267abd31393)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:15.820162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:15.841885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:15.850334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b7bec530af43caa7a00908c89ca482'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.851167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b7bec530af43caa7a00908c89ca482'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.872662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b7bec530af43caa7a00908c89ca482'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.883219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1972f975d50140168007d267abd31393'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.884920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b7bec530af43caa7a00908c89ca482'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.885782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1277cb0b-fa7c-4880-afc3-ed2618275e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b7bec530af43caa7a00908c89ca482'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.890519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1972f975d50140168007d267abd31393'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.903431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 792/1581] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:15 2022] DELETE /identity/v3/users/13b7bec530af43caa7a00908c89ca482 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:15.904743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1972f975d50140168007d267abd31393'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.906525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1972f975d50140168007d267abd31393'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.907761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:15.908046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:15.908362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:15.915152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1972f975d50140168007d267abd31393'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:15.922749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 790/1582] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:15 2022] DELETE /identity/v3/users/1972f975d50140168007d267abd31393 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:15.933134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:15.934339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.004848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 793/1583] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:16.025441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.027896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.136808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b952e3-f408-4304-8fb4-450dc0b515a3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.141582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.142406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.142680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.142973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.154589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.157279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4ab99-e48e-4da0-baa3-f398f5f1540e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.167999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 791/1584] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:16.323198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.324881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.338711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e93f02-3a87-4705-9844-5d079a45dbd1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.344255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.345125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.345570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.346056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] PATH_INFO: `/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.347416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d1b7b08197445fbeadb3f63c5b54e8)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.349538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.419355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d1b7b08197445fbeadb3f63c5b54e8', 'request_id': 'req-2e1f62ed-a849-4afa-a927-5ee4c7873d38'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:16.427950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d1b7b08197445fbeadb3f63c5b54e8', 'request_id': 'req-2e1f62ed-a849-4afa-a927-5ee4c7873d38'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:16.441980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d1b7b08197445fbeadb3f63c5b54e8', 'request_id': 'req-2e1f62ed-a849-4afa-a927-5ee4c7873d38'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:16.465871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] The token cache is being invalidate because project f1d1b7b08197445fbeadb3f63c5b54e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:16.466257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:16.467031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2e1f62ed-a849-4afa-a927-5ee4c7873d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:16.469030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 794/1585] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:16 2022] DELETE /identity/v3/projects/f1d1b7b08197445fbeadb3f63c5b54e8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:16.475172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.475838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.475838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.499957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:16.501320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.569795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 792/1586] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:16.579173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeccec-4437-4110-9822-2007673179b2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.579749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeccec-4437-4110-9822-2007673179b2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.579749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeccec-4437-4110-9822-2007673179b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.590704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:16.606420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fafeccec-4437-4110-9822-2007673179b2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:16.607680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fafeccec-4437-4110-9822-2007673179b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.628291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 795/1587] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:16.628584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.639565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906afe13-1dc6-4bba-b154-5e0fbd3e496c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.644557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.645392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.645677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.645773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.646477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:16.646849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.648762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.662559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 793/1588] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:28:16.667336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafeccec-4437-4110-9822-2007673179b2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.676444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafeccec-4437-4110-9822-2007673179b2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.682157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.682852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.683132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.683403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.684051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.686272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.718201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 796/1589] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:16 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:16.722390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.742680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b21fb-c4e1-4d4b-bfaa-b3eea1a58072 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.769582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:16.862828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.863513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.863844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.864153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.866618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.871030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.912386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.913111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:16.928886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 794/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:16 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:16.930055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdcac4-0f8e-4362-be22-85086d0b5d16 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.934229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:16.943307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.943825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.944315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.944558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] PATH_INFO: `/v3/projects/aab762db6e654d0b8e4ed73dd80117e6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.946886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab762db6e654d0b8e4ed73dd80117e6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.950392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.958859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6d53-0613-4707-a8df-61c68a78d05f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:16.966610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:16.967819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:16.967949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:16.968388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:16.969205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:16.971136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:16.985989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 795/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:16.990254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.002512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4b80-8d63-4658-a352-98fac31e01bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.016936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab762db6e654d0b8e4ed73dd80117e6', 'request_id': 'req-dcaebc5e-dddb-446a-ac88-bef397539145'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:17.027163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab762db6e654d0b8e4ed73dd80117e6', 'request_id': 'req-dcaebc5e-dddb-446a-ac88-bef397539145'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:17.043549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab762db6e654d0b8e4ed73dd80117e6', 'request_id': 'req-dcaebc5e-dddb-446a-ac88-bef397539145'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:17.059107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] The token cache is being invalidate because project aab762db6e654d0b8e4ed73dd80117e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:17.059453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:17.060482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-dcaebc5e-dddb-446a-ac88-bef397539145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:17.067043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 797/1592] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:16 2022] DELETE /identity/v3/projects/aab762db6e654d0b8e4ed73dd80117e6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:17.068750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca39c1c-7ae8-456e-918c-967af3e12079 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.069042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca39c1c-7ae8-456e-918c-967af3e12079 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.069318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca39c1c-7ae8-456e-918c-967af3e12079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.100722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dca39c1c-7ae8-456e-918c-967af3e12079 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:17.102359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dca39c1c-7ae8-456e-918c-967af3e12079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:17.182781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 798/1593] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:17.185263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.185618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.186903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.212473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:17.213435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.213844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:17.214243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.214621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.215065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.215894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.217817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.223499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 799/1594] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:17.227905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.233580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 796/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:17.237392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.242107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:17.243737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07c43-bb35-44d5-9855-465d8f86aae2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.249080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.249990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.249990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.250201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.250851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:17.251149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.252797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.262337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851597d-844c-47e2-aabf-a7cbd1b3486f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.265843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 800/1596] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:28:17.269918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.269918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.269918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.269918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] PATH_INFO: `/v3/projects/8bcd746102674cb9b173c74eb0bc3686/users/862c22555d5b44f7a0566553fc13ce5c/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.271267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.282243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bf506-f766-4a9f-997e-bde6330a1d05 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.288706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.289683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.290110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.290559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.290815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcd746102674cb9b173c74eb0bc3686, user_id=862c22555d5b44f7a0566553fc13ce5c, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.291683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.293750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.293887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc2991-a69b-454e-8485-4e7fb3952c54 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.313591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 797/1597] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:17 2022] PUT /identity/v3/projects/8bcd746102674cb9b173c74eb0bc3686/users/862c22555d5b44f7a0566553fc13ce5c/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:17.324756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.325196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.325568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.337881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 801/1598] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:17.339688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.372000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] MFA Rules not processed for user `862c22555d5b44f7a0566553fc13ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:17.373611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:17.374391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:17.435384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccb246-6b1c-4267-8bf4-1488ded2a232 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.450509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.450509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.450509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.450509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.450883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.451401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.487089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:17.492287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 802/1599] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:17.493962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 798/1600] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:17.499865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.502029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.534436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed6730-a625-4212-98d1-4e395d387b1e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.535065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa8bc6-39d2-4e61-940d-c063401f2a7b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.549396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.550305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.550685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.551126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.551487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.551714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.552197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.552512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.552778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.553486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.553700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.565416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.567343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 803/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:17.571361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.582706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4384a-27e4-4e2e-a1f3-18bb342bf003 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.594446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.595079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.595351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.595617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.596734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.600104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.616317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 799/1602] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:17.627716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.629466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 804/1603] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:17.634114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.663429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b43cc-475e-4e2e-a485-418add276eab admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.776242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617576fd-98a7-4f5e-964f-6f938667f44b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.786761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.788449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.789035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.789507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.790408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.793292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.813163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.814793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.818204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.819416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] PATH_INFO: `/v3/projects/61e657b5d3334a42a87ceebe86270698/users/7d30121b50534b09b01e81894898d911/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.827795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 800/1604] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:17.829283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.841062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6af24-7f7e-4709-bdb9-1248dd1b1632 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.846216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:17.846988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.847336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.847841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.848602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.851005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.863769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e657b5d3334a42a87ceebe86270698, user_id=7d30121b50534b09b01e81894898d911, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:17.864971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 801/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:17.872166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:17.875360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ca0a7-7aba-452b-a1b8-3f267a337f2a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:17.890798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c8bb-ff3c-42fe-af75-0483489a006a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:17.896884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 805/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:17 2022] PUT /identity/v3/projects/61e657b5d3334a42a87ceebe86270698/users/7d30121b50534b09b01e81894898d911/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:17.915011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:17.915698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:17.916344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:17.959977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] MFA Rules not processed for user `7d30121b50534b09b01e81894898d911`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:17.962681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:18.009580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:18.009580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.009580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.009580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] PATH_INFO: `/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80/users/47e7a350f783460abf8c2755e27b1b12/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.054654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:18.064283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244cdc1f16fb4aeb9bf1d287aa7c6e80, user_id=47e7a350f783460abf8c2755e27b1b12, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:18.074408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:18.074545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 806/1607] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:18.097563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:18.099225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:18.123662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 802/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:17 2022] PUT /identity/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80/users/47e7a350f783460abf8c2755e27b1b12/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:18.127832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:18.217472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:18.232416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7971-7315-431f-8d9e-fa8ece0a58f0 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:18.253309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0590e7-a770-4fa4-b656-69f50365a6e5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:18.259506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:18.260168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.260517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.260704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.261357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:18.263143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:18.275722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 803/1609] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:18.280164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:18.291013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc520-b86b-4e4e-b1a7-7839c958b14b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:18.298092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:18.298847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.299228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.299573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] PATH_INFO: `/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80/users/47e7a350f783460abf8c2755e27b1b12/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.301626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=244cdc1f16fb4aeb9bf1d287aa7c6e80, user_id=47e7a350f783460abf8c2755e27b1b12, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:18.302947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:18.303624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.303983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.304346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.304654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f8c04-0425-49af-a113-1966e4c0c3fb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:18.317849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 804/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:18 2022] PUT /identity/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80/users/47e7a350f783460abf8c2755e27b1b12/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:18.328389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.328668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.328935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.364038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] MFA Rules not processed for user `47e7a350f783460abf8c2755e27b1b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:18.366368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:18.376081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:18.377712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:18.390056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 807/1611] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:28:18.458514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 805/1612] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:18.470712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:18.472691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:18.485167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0baa-74ba-4c85-a8f5-955165aa8e14 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:18.542247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:18.543842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:18.544308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:18.544910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:18.573349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:18.623596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:18.625154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:18.636148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 808/1613] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:27.871455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:27.882140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbf749-77cc-4e10-841d-f58f8d0de651 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:27.886134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:27.886773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:27.887095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:27.887368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:27.897797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:27.899465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e6c0-5462-4d5c-a868-d585f906913d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:27.910239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 806/1614] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:28.356973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:28.426230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15880ae4-af59-4fbd-92dd-91076d82efd3 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:28.431285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:28.432093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:28.432213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:28.432491 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] PATH_INFO: `/v3/users/7d30121b50534b09b01e81894898d911` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:28.433709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d30121b50534b09b01e81894898d911)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:28.435637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:28.489281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d30121b50534b09b01e81894898d911'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:28.490438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d30121b50534b09b01e81894898d911'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:28.509137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d30121b50534b09b01e81894898d911'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:28.522120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d30121b50534b09b01e81894898d911'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:28.522963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d30121b50534b09b01e81894898d911'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:28.537985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 809/1615] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:28 2022] DELETE /identity/v3/users/7d30121b50534b09b01e81894898d911 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:28.542642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:28.542922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:28.543189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:28.562053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:28.563224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:28.630949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 807/1616] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:28.640998 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:28.642736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:28.690120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:28.704081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81558c22-74a0-4808-9e22-00a40fa5ec2e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:28.709071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:28.709862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:28.710402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:28.710959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:28.734024 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:28.735482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6b296-09ed-43e8-9d7b-a0ea1cf536a0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:28.746620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 810/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:29.038296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.039882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:29.053717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddc3f7-33ff-4a4b-81ce-f8bc957246fd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.059256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.060694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.061046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.061458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] PATH_INFO: `/v3/projects/61e657b5d3334a42a87ceebe86270698` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.062716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e657b5d3334a42a87ceebe86270698)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.064756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.101088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:29.115272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e657b5d3334a42a87ceebe86270698', 'request_id': 'req-ae552f97-f1c6-4cda-b4d8-952380be89c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:29.121496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e657b5d3334a42a87ceebe86270698', 'request_id': 'req-ae552f97-f1c6-4cda-b4d8-952380be89c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:29.130310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e657b5d3334a42a87ceebe86270698', 'request_id': 'req-ae552f97-f1c6-4cda-b4d8-952380be89c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:29.142665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] The token cache is being invalidate because project 61e657b5d3334a42a87ceebe86270698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:29.142948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:29.143718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ae552f97-f1c6-4cda-b4d8-952380be89c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:29.145385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 808/1618] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:29 2022] DELETE /identity/v3/projects/61e657b5d3334a42a87ceebe86270698 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:29.150510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.150791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.151065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.171730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:29.172971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:29.236449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 811/1619] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:29.245375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.245657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.245929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.265738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:29.266933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:29.276654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 809/1620] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:29.281047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.291428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7b890-e997-4c79-ba15-baee4df46d90 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.296553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.297166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.297502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.297909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.298514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:29.298819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.300422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.312632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 812/1621] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:29.316986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.326435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3eb18-55fc-4a9a-a09a-b9cb22b18a8e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.331406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.332038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.332349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.332617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.333254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.335161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.368192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 810/1622] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:29.370615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.380874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1598e0-8a8c-476f-9f00-40ffd9bfcbd3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.440011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.440644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.440921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.441187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.441817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.443750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.468784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 813/1623] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:29.472854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.483213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918de7c3-50b5-4907-b1bd-568eed0c5530 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.488001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.488637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.488913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.489182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.489833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.491412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.502367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 811/1624] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:29.506439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.516736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22330e4-8c03-41f4-97d0-29fed8eb502e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.521871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.522472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.522746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.523217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.523722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.525354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab92ae-cf71-45e2-aff8-14c451d5fdc6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.536740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 814/1625] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:29.541121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.551261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affb0b-e21e-4c9f-84c3-d9875817c229 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.555825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.556510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.556787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.557054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] PATH_INFO: `/v3/projects/5a8eb37778e240b2a1f342082426b677/users/666984bd91ae4627b3a3a46a82058592/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.571914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8eb37778e240b2a1f342082426b677, user_id=666984bd91ae4627b3a3a46a82058592, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.574744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.592471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 812/1626] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:29 2022] PUT /identity/v3/projects/5a8eb37778e240b2a1f342082426b677/users/666984bd91ae4627b3a3a46a82058592/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:29.603018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.603302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.603575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.632639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] MFA Rules not processed for user `666984bd91ae4627b3a3a46a82058592`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:29.633839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:29.676074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:29.701081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 815/1627] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:29.705764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.769407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f209549-66c0-469c-9af8-3d367cdb0260 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.774183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.774792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.775072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.775339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.776005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.777972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.811777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 813/1628] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:29.813231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.824584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb98944-56e5-4dbe-886e-8162b1db6143 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.885145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.885673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.885949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.886217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.886866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.888827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.911817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 816/1629] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:29 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:29.916074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.927059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d5827-2e62-448f-b335-c402a0f36389 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.933592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.934249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.934529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.934800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.935457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.937136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.948547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 814/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:29.952626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:29.965293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5b89-c0c1-4ac5-9fbb-a6a4f4167f9e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:29.970913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:29.971534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:29.971829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:29.972107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:29.972812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:29.974429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8a4b3-9eaf-4e29-ae46-ed16a85ef39a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:29.984997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 817/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:29.990827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.001292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16e54-7f65-468a-99ad-b94fa806beb1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.006408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.007072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.007368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.007730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] PATH_INFO: `/v3/projects/52b59f185f764fb8bfb044783e8a80fa/users/cfe4c60111d1409eb825cab0a63d9f18/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.022882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b59f185f764fb8bfb044783e8a80fa, user_id=cfe4c60111d1409eb825cab0a63d9f18, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.025567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.038639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 815/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:29 2022] PUT /identity/v3/projects/52b59f185f764fb8bfb044783e8a80fa/users/cfe4c60111d1409eb825cab0a63d9f18/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:30.048991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.049280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.049548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.078989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] MFA Rules not processed for user `cfe4c60111d1409eb825cab0a63d9f18`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:30.080235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:30.149809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 818/1633] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:30.154593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.160147 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:30.221505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e528a6-58bf-4e48-9085-3eda9ac2244a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.226672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.227283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.227564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.227834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.228503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.230433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.258966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 816/1634] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:30 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:30.263248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.274578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512accc9-7b9c-4a3a-8e4f-861cdd4885fe None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.326224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:30.339648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.340305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.340588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.340865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.341524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.343430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.368255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 819/1635] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:30 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:30.369431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.380836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c14af-4274-47fe-942d-4e784f15b00e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.385631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.386244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.386577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.386820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.387520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.389305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.401588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 817/1636] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:30.406134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.416496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce945f-7a44-4e07-b69c-db45e703ffc8 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.422110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.422893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.423161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.423434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] PATH_INFO: `/v3/projects/16ac85b78300466c89073b9a186d98b3/users/3d8f7f18f15346789998e3777cb880d8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.439633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ac85b78300466c89073b9a186d98b3, user_id=3d8f7f18f15346789998e3777cb880d8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.442337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.465105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 820/1637] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:30 2022] PUT /identity/v3/projects/16ac85b78300466c89073b9a186d98b3/users/3d8f7f18f15346789998e3777cb880d8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:30.469320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.479695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ed72-aafa-4609-95c3-21e0f067497b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.540597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.541221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.541519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.541782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.542426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.544033 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92568fa-b0a7-4353-9909-14df3866d2d0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.558265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 818/1638] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:28:30.559080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.570128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107536e-cd6e-45d7-911b-53f1f9728682 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.575036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.575720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.575990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.576284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] PATH_INFO: `/v3/projects/16ac85b78300466c89073b9a186d98b3/users/3d8f7f18f15346789998e3777cb880d8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.578206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ac85b78300466c89073b9a186d98b3, user_id=3d8f7f18f15346789998e3777cb880d8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.580869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.591483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 821/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:30 2022] PUT /identity/v3/projects/16ac85b78300466c89073b9a186d98b3/users/3d8f7f18f15346789998e3777cb880d8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:30.602703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a947-951d-454a-9f84-b335934c918c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.602982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a947-951d-454a-9f84-b335934c918c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.603245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5a947-951d-454a-9f84-b335934c918c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.630337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-27f5a947-951d-454a-9f84-b335934c918c None None] MFA Rules not processed for user `3d8f7f18f15346789998e3777cb880d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:30.631555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27f5a947-951d-454a-9f84-b335934c918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:30.706278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:30.710655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 819/1640] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:28:30.721804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:30.723529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:30.783881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e5b7b-c645-4241-9158-edaf5f6086a4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:30.787922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:30.788584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:30.788958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:30.789265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:30.847690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:30.849133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:30.859477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 822/1641] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:36.641502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5a947-951d-454a-9f84-b335934c918c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:36.651384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5a947-951d-454a-9f84-b335934c918c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:36.655225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:36.655815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:36.656141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:36.656436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:36.712817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:36.714256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:36.723500 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 820/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:37.388161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:37.397503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751bcc4-0703-44cc-8e7a-4e6813da1e3e service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:37.401912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:37.402520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:37.402796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:37.403063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:37.412972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:37.414372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1aef1-0c93-45c4-b6ec-efa4e2932f73 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:37.424261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 823/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:46.720102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=28 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:47.419962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:55.136100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:55.201455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed31e8-9ccc-48ca-8d1c-60c2f391008f service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:55.206610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:55.207545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:55.207545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:55.207831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] PATH_INFO: `/v3/users/666984bd91ae4627b3a3a46a82058592` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:55.208983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666984bd91ae4627b3a3a46a82058592)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:55.211226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:55.263740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666984bd91ae4627b3a3a46a82058592'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.272736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666984bd91ae4627b3a3a46a82058592'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.290345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666984bd91ae4627b3a3a46a82058592'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.291116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666984bd91ae4627b3a3a46a82058592'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.301837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666984bd91ae4627b3a3a46a82058592'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.308404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 821/1644] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:55 2022] DELETE /identity/v3/users/666984bd91ae4627b3a3a46a82058592 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:55.312943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:55.313428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:55.313532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:55.332603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:55.333722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:55.402208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 824/1645] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:55.412879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:55.414556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:55.477124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bb181-ee05-4839-80e5-b5a751813f45 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:55.481138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:55.481795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:55.482073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:55.482503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:55.491545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:55.493009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:55.502538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 822/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:28:55.764767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:55.766226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:55.784456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184ae79-0b72-489e-9eda-9bb506d24c8b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:55.790310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:55.790986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:55.791227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:55.791675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] PATH_INFO: `/v3/projects/5a8eb37778e240b2a1f342082426b677` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:55.793034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8eb37778e240b2a1f342082426b677)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:55.794944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:55.843487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8eb37778e240b2a1f342082426b677', 'request_id': 'req-e003c365-dd64-4579-a6f5-867124499e1a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.850319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8eb37778e240b2a1f342082426b677', 'request_id': 'req-e003c365-dd64-4579-a6f5-867124499e1a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.860630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8eb37778e240b2a1f342082426b677', 'request_id': 'req-e003c365-dd64-4579-a6f5-867124499e1a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.872222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] The token cache is being invalidate because project 5a8eb37778e240b2a1f342082426b677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:55.872511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.873165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:55.874723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 825/1647] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:55 2022] DELETE /identity/v3/projects/5a8eb37778e240b2a1f342082426b677 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:28:55.878857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:55.888487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f663c7-9c6d-41a9-ac59-c97dbe5a9b60 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:55.906974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:55.947977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:55.948732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:55.949009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:55.949312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] PATH_INFO: `/v3/users/cfe4c60111d1409eb825cab0a63d9f18` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:55.950647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe4c60111d1409eb825cab0a63d9f18)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:55.952657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:56.001094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe4c60111d1409eb825cab0a63d9f18'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.008902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe4c60111d1409eb825cab0a63d9f18'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.023051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe4c60111d1409eb825cab0a63d9f18'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.023917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe4c60111d1409eb825cab0a63d9f18'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.037841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe4c60111d1409eb825cab0a63d9f18'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.041484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 823/1648] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:55 2022] DELETE /identity/v3/users/cfe4c60111d1409eb825cab0a63d9f18 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:56.353066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:56.354757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:56.376775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003c365-dd64-4579-a6f5-867124499e1a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:56.441564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:56.442211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:56.442487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:56.442753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] PATH_INFO: `/v3/projects/52b59f185f764fb8bfb044783e8a80fa` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:56.443867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b59f185f764fb8bfb044783e8a80fa)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:56.445797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:56.479203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:56.498299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b59f185f764fb8bfb044783e8a80fa', 'request_id': 'req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.504899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b59f185f764fb8bfb044783e8a80fa', 'request_id': 'req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.512383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b59f185f764fb8bfb044783e8a80fa', 'request_id': 'req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.526315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] The token cache is being invalidate because project 52b59f185f764fb8bfb044783e8a80fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:56.526606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.527282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.528831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 826/1649] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:56 2022] DELETE /identity/v3/projects/52b59f185f764fb8bfb044783e8a80fa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:56.533003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:56.534399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:56.547102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014dadf9-582e-4168-95d8-88c61d6d1625 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:56.618474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:56.619377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:56.619701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:56.619971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] PATH_INFO: `/v3/users/3d8f7f18f15346789998e3777cb880d8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:56.621140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8f7f18f15346789998e3777cb880d8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:56.623103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:56.669935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:56.673051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8f7f18f15346789998e3777cb880d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.681117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8f7f18f15346789998e3777cb880d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.694575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8f7f18f15346789998e3777cb880d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.695251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8f7f18f15346789998e3777cb880d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.703708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3affcf5e-8920-4406-a79d-41fe83424b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8f7f18f15346789998e3777cb880d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:56.709895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 824/1650] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:28:56 2022] DELETE /identity/v3/users/3d8f7f18f15346789998e3777cb880d8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:56.997574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:56.999345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:57.014192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d5ab-6c4b-421d-9bd8-b88857c0fd1c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.071494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.072156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.072433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.072805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] PATH_INFO: `/v3/projects/16ac85b78300466c89073b9a186d98b3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.073917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ac85b78300466c89073b9a186d98b3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.075803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.124595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ac85b78300466c89073b9a186d98b3', 'request_id': 'req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:57.131057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ac85b78300466c89073b9a186d98b3', 'request_id': 'req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:57.139914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ac85b78300466c89073b9a186d98b3', 'request_id': 'req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:57.150592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] The token cache is being invalidate because project 16ac85b78300466c89073b9a186d98b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:28:57.150875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:57.151826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b8c366fc-e29f-40a6-a2dd-79e5fbf7724d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:28:57.153647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 827/1651] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:28:56 2022] DELETE /identity/v3/projects/16ac85b78300466c89073b9a186d98b3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:57.158245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.158339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.158694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.180805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:57.182085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:57.242604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 825/1652] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:57.250420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97504-e276-4f11-8990-509a4b035683 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.250702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97504-e276-4f11-8990-509a4b035683 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.250979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97504-e276-4f11-8990-509a4b035683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.272604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-46b97504-e276-4f11-8990-509a4b035683 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:57.273840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-46b97504-e276-4f11-8990-509a4b035683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:57.282989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 828/1653] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:57.287240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.297450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b589e-eb1d-44db-8f36-70a5e6686507 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.302194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.302798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.303072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.303336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.303977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:28:57.304303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.305877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.317741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 826/1654] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:28:57.322182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b97504-e276-4f11-8990-509a4b035683 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.332089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b97504-e276-4f11-8990-509a4b035683 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.337384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.338003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.338337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.338714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.339366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.341302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.381991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 829/1655] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:28:57 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:28:57.383133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.393569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8474c-7e94-4527-a88d-02ece6f3e44c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.489132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.489879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.491173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.491545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.492985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.495683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.545646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 827/1656] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:28:57 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:28:57.546479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.558794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a496-1cc7-41ea-ad3a-fd0f42a05023 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.563889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.565038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.565669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.565990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.566846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.568549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.582025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 830/1657] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:57.586681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.596628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51963d95-7e97-4e88-9152-393b740971ec admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.601232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.601912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.602300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.602577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.603306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.605057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7982b6-c2fe-4810-9227-7f84ad41238c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.617035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 828/1658] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:28:57.621372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.629027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:57.632960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e9c7-89df-4966-ba2a-3a2aedbcab83 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.639861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.640938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.641402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.642236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] PATH_INFO: `/v3/projects/5d8e73317ddd440a97287c6c745bb387/users/96dd9688c16a41948e69c0a5fbfa3d88/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.658593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8e73317ddd440a97287c6c745bb387, user_id=96dd9688c16a41948e69c0a5fbfa3d88, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.661261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.675854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 831/1659] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:28:57 2022] PUT /identity/v3/projects/5d8e73317ddd440a97287c6c745bb387/users/96dd9688c16a41948e69c0a5fbfa3d88/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:28:57.687118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fca6-2dde-49a3-92cd-b8d3890ac694 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.687414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fca6-2dde-49a3-92cd-b8d3890ac694 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.687708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fca6-2dde-49a3-92cd-b8d3890ac694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.715287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-df75fca6-2dde-49a3-92cd-b8d3890ac694 None None] MFA Rules not processed for user `96dd9688c16a41948e69c0a5fbfa3d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:28:57.716531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-df75fca6-2dde-49a3-92cd-b8d3890ac694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:57.729608 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:28:57.783724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 829/1660] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:28:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:28:57.795692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:28:57.797418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:28:57.854814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b259b-93e6-4d30-a202-140a5d402355 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:28:57.858712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:28:57.859327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:28:57.859600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:28:57.859869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:28:57.868572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:28:57.870015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a4af6-583d-4eac-ac92-9b43002a0110 service glance] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:28:57.878885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 832/1661] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:28:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:00.866509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:00.866967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:00.867199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:00.868870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 830/1662] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:29:00.881136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:00.881772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:00.881870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:00.905933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:00.907517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:00.945652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:00.947613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:00.998361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 833/1663] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:29:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:01.022519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.024179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e5f97-251c-4833-bef9-dda7d32e0928 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.024333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:01.028940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.029613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.029884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.030155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] PATH_INFO: `/v3/users/e4e126b005e149258013ce3b6cf15524` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.031292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e126b005e149258013ce3b6cf15524)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.033849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.038641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a408a-0949-46c6-84ef-a0cf89f02603 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.045181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.045806 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.046080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.046346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.056691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.058402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.068945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 834/1664] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:01.088802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e126b005e149258013ce3b6cf15524'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.099234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e126b005e149258013ce3b6cf15524'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.113000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e126b005e149258013ce3b6cf15524'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.113792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e126b005e149258013ce3b6cf15524'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.121987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9c65858b-3e77-46bc-82e2-52d98b02ae73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e126b005e149258013ce3b6cf15524'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.128339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 831/1665] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:00 2022] DELETE /identity/v3/users/e4e126b005e149258013ce3b6cf15524 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:01.306393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.317739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e264050-9345-40d2-994e-3635cce128c4 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.374861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.375500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.375773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.376040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] PATH_INFO: `/v3/projects/2adf8ac988764dc29c6f0442b2398124` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.377286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adf8ac988764dc29c6f0442b2398124)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.379237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.389988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:01.435164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adf8ac988764dc29c6f0442b2398124', 'request_id': 'req-2031f8d1-ea40-4179-85fa-6d59389c5d95'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.445104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adf8ac988764dc29c6f0442b2398124', 'request_id': 'req-2031f8d1-ea40-4179-85fa-6d59389c5d95'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.455143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adf8ac988764dc29c6f0442b2398124', 'request_id': 'req-2031f8d1-ea40-4179-85fa-6d59389c5d95'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.466210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] The token cache is being invalidate because project 2adf8ac988764dc29c6f0442b2398124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:01.466492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.467128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2031f8d1-ea40-4179-85fa-6d59389c5d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:01.468805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 835/1666] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:01 2022] DELETE /identity/v3/projects/2adf8ac988764dc29c6f0442b2398124 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:01.473843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028615-9187-488f-8326-4d4e458a936f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.474126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028615-9187-488f-8326-4d4e458a936f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.474556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028615-9187-488f-8326-4d4e458a936f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.494839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6c028615-9187-488f-8326-4d4e458a936f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:01.496193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6c028615-9187-488f-8326-4d4e458a936f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:01.527344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:01.558111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 832/1667] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:01.565838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.566124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.566395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.586996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:01.588214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:01.596452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 836/1668] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:01.600983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c028615-9187-488f-8326-4d4e458a936f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.611880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c028615-9187-488f-8326-4d4e458a936f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.617213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.618082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.618325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.618601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.619259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:01.619606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.621341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.633944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 833/1669] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:01.638848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.648669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c723f-8f29-4628-b4ab-1b233f80a2d1 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.653780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.654387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.654669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.654943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.655577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.657600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.688085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 837/1670] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:01 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:01.692257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.703425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f34a-7c09-4637-a4db-ab3d5f0556ca admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.762155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.762778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.763055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.763325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.763977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.765939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.789559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 834/1671] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:01 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:01.793424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.805899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89aea9-5237-4951-a36b-1515642913b3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.810576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.811190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.811464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.811729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.812439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.814063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.824434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 838/1672] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:01.828917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.840173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39273caa-8f74-4ca0-b505-a87e1889d408 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.845737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.846365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.846670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.846936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.847605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.849298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3b65e-01d8-4ae2-b8c7-1df268959c50 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.860848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 835/1673] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:01.865283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:01.875981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbc11-4dd9-4b93-9317-7b2741787991 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:01.880818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:01.881777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.882437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.882437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] PATH_INFO: `/v3/projects/3715866a8e3b448284724cea72a3055e/users/34aff971c9ac4db397b1dd3a3833f485/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.900074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3715866a8e3b448284724cea72a3055e, user_id=34aff971c9ac4db397b1dd3a3833f485, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:01.903161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:01.918880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 839/1674] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:01 2022] PUT /identity/v3/projects/3715866a8e3b448284724cea72a3055e/users/34aff971c9ac4db397b1dd3a3833f485/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:01.930709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:01.931299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:01.931299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:01.962763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] MFA Rules not processed for user `34aff971c9ac4db397b1dd3a3833f485`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:01.964010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:02.032638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 836/1675] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:02.037387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.104817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a2f93-21f2-4025-9c63-dfcd1ca5a1e7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.110523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.111148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.111425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.111699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.112381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.114291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.145666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 840/1676] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:02 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:02.150137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.161070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1fffc-96ac-4121-84d6-3ccdcf22f1b0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.190960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:02.220625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.221509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.221782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.222054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.222737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.224685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.249832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 837/1677] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:02 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:02.251278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.262035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af0a78-dbf2-436f-bf08-f2bc36ee49e6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.267013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.267624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.267902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.268202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.268852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.270447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.281515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 841/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:02.286146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.296689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cab68-3357-41e6-a633-62e38d78654c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.301365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.302136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.302411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.302721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] PATH_INFO: `/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758/users/5ce24c4d355d4305b3f1d206f9fa50bf/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.317830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28fd7b22d1d4188a3aaed9b29ea4758, user_id=5ce24c4d355d4305b3f1d206f9fa50bf, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.320503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.338755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 838/1679] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:02 2022] PUT /identity/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758/users/5ce24c4d355d4305b3f1d206f9fa50bf/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:02.344077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.354385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aab13-fb48-43e0-a1f0-0fa4af0d6c5c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.358148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:02.414795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.415414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.415688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.415951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.416690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.418275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5b1aa-9614-4997-a95b-94145dbcc5f4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.428174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 842/1680] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:02.434598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.449921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eba5fc-df89-4209-b933-724766049117 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.455614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.456317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.456599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.456863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] PATH_INFO: `/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758/users/5ce24c4d355d4305b3f1d206f9fa50bf/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.459001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28fd7b22d1d4188a3aaed9b29ea4758, user_id=5ce24c4d355d4305b3f1d206f9fa50bf, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.461691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.473149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 839/1681] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:02 2022] PUT /identity/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758/users/5ce24c4d355d4305b3f1d206f9fa50bf/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:02.483657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.483941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.484280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.516201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] MFA Rules not processed for user `5ce24c4d355d4305b3f1d206f9fa50bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:02.517752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:02.597102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 843/1682] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:02.608974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.610695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:02.670039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26006-683d-4969-a97c-b70e392dc00b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.673690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.674294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.674592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.674963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.683950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.685432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.694754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 840/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:02.773005 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.787526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe69b3-f476-4742-ba2e-baa93bf8d9d3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.793871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.794494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.794755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.795022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.862279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.862279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fdb5e-0b06-4113-95f0-345e723c4e9e service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:02.870441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 844/1684] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:02.882938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:02.884381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:02.970751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ca92-68c7-43a2-b0ae-ad819edcfaf4 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:02.979322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:02.979980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:02.980294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:02.980564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] PATH_INFO: `/v3/users/998b037068874210a1e13b98e0d8ad5b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:02.981693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998b037068874210a1e13b98e0d8ad5b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:02.983602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:03.029470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:03.056048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998b037068874210a1e13b98e0d8ad5b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.062976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998b037068874210a1e13b98e0d8ad5b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.076566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998b037068874210a1e13b98e0d8ad5b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.077427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998b037068874210a1e13b98e0d8ad5b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.085101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998b037068874210a1e13b98e0d8ad5b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.091464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 841/1685] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:02 2022] DELETE /identity/v3/users/998b037068874210a1e13b98e0d8ad5b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:03.096220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.096714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.096809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.116971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:03.118243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:03.139255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:03.179267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 845/1686] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:03.189505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:03.191081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:03.253030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238ed4-4130-4e6b-b1b9-aed11dc3faf5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:03.256855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:03.257500 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.257783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.258053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.269104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:03.270966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6b6b-d507-4c04-b77e-11f21e441759 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:03.284568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 842/1687] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:03.605372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:03.620611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:03.620611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5938a9-2f71-4501-b3dc-bc9baeac3592 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:03.624325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:03.624953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.625235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.625507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] PATH_INFO: `/v3/projects/951180f93a5e4676b5294930b163a71e` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.626639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951180f93a5e4676b5294930b163a71e)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:03.628616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:03.682344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951180f93a5e4676b5294930b163a71e', 'request_id': 'req-91266051-c596-4ee3-b4de-869266c362c3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.691002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951180f93a5e4676b5294930b163a71e', 'request_id': 'req-91266051-c596-4ee3-b4de-869266c362c3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.701304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951180f93a5e4676b5294930b163a71e', 'request_id': 'req-91266051-c596-4ee3-b4de-869266c362c3'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.712421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] The token cache is being invalidate because project 951180f93a5e4676b5294930b163a71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:03.712709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.713395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-91266051-c596-4ee3-b4de-869266c362c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:03.714909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 846/1688] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:03 2022] DELETE /identity/v3/projects/951180f93a5e4676b5294930b163a71e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:03.720311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.720601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.720876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.740246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:03.741472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:03.802228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 843/1689] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:03.809966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea298c-b293-4e93-a958-1775afde834e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.810252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea298c-b293-4e93-a958-1775afde834e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.810484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea298c-b293-4e93-a958-1775afde834e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.831401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-19ea298c-b293-4e93-a958-1775afde834e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:03.832651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-19ea298c-b293-4e93-a958-1775afde834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:03.841173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 847/1690] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:03.845457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:03.855651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c539211-72d8-45d8-aaf4-902b26f0cd9d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:03.860561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:03.861168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.861447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.861716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.862361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:03.862718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:03.864424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:03.875946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 844/1691] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:03.880021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea298c-b293-4e93-a958-1775afde834e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:03.890307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea298c-b293-4e93-a958-1775afde834e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:03.896129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:03.896897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:03.897202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:03.897475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:03.898214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:03.900376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:03.934992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 848/1692] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:03 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:03.939324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:03.949648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d61bc7-be11-4f95-8e05-f49e7bacdcfa admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:03.982119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:04.010421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.011047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.011312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.011584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.012242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.014164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.037790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 845/1693] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:03 2022] POST /identity/v3/users => generated 278 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:04.042026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.052615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371fceb-b280-4203-add2-80ff0dff19db admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.057647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.058272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.058967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.058967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.059560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.061190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.071866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 849/1694] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:04.076062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.094425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a6baa-be1f-4676-848b-35088e4e9574 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.099750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.100490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.100821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.101160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.101823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.103381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee06564-ce5b-46da-9c58-6e553e4bca8f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.135115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 846/1695] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:04.139533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.154867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150598-02ed-4496-ae63-bc7adf882a48 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.161212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.161943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.162248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.162517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] PATH_INFO: `/v3/projects/a9a4ac71270f4ae5bf46a96b837bc82a/users/cd17a6920c9e4fcc8c176d72ce615257/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.186627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a4ac71270f4ae5bf46a96b837bc82a, user_id=cd17a6920c9e4fcc8c176d72ce615257, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.189745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.209769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 850/1696] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:04 2022] PUT /identity/v3/projects/a9a4ac71270f4ae5bf46a96b837bc82a/users/cd17a6920c9e4fcc8c176d72ce615257/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:04.222203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.223102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.223522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.261953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] MFA Rules not processed for user `cd17a6920c9e4fcc8c176d72ce615257`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:04.263145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:04.339912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 847/1697] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:04.417462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.421322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:04.486734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:04.495015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17437-0fb5-41ee-812e-0626e3bc65c7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.504392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.504392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.504392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.504392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.518189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.520348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248701e-5e55-43aa-8a89-438f14e3461c service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.530167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 851/1698] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:04.546755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.636763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9867e5-9f7c-4868-9912-c9d4db629698 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.643213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.644472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.644472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.644653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] PATH_INFO: `/v3/users/cd17a6920c9e4fcc8c176d72ce615257` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.646039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd17a6920c9e4fcc8c176d72ce615257)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.648100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.696574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:04.698390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd17a6920c9e4fcc8c176d72ce615257'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:04.707986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd17a6920c9e4fcc8c176d72ce615257'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:04.719928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd17a6920c9e4fcc8c176d72ce615257'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:04.720765 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd17a6920c9e4fcc8c176d72ce615257'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:04.728926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd17a6920c9e4fcc8c176d72ce615257'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:04.741988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 848/1699] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:04 2022] DELETE /identity/v3/users/cd17a6920c9e4fcc8c176d72ce615257 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:04.743451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.743811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.744226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.764134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:04.765455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:04.841116 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 852/1700] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:04.850959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:04.852430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:04.916576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440df2d5-7b68-4dbf-ad21-7822183f657a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:04.920867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:04.921449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:04.921762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:04.922032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:04.932018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:04.933566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45395203-df6e-4e49-8d8f-db0468d633d4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:04.943555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 849/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:05.222260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.225042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:05.242683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7625d-7491-424a-8164-95f6c2613031 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.247922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.248687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.248971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.249240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] PATH_INFO: `/v3/projects/a9a4ac71270f4ae5bf46a96b837bc82a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.250420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a4ac71270f4ae5bf46a96b837bc82a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.252412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.300459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a4ac71270f4ae5bf46a96b837bc82a', 'request_id': 'req-a816ef5b-da71-4662-ad95-0dec53aebed2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:05.306681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a4ac71270f4ae5bf46a96b837bc82a', 'request_id': 'req-a816ef5b-da71-4662-ad95-0dec53aebed2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:05.315046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a4ac71270f4ae5bf46a96b837bc82a', 'request_id': 'req-a816ef5b-da71-4662-ad95-0dec53aebed2'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:05.325278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] The token cache is being invalidate because project a9a4ac71270f4ae5bf46a96b837bc82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:05.325556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:05.326328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a816ef5b-da71-4662-ad95-0dec53aebed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:05.327931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 853/1702] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:05 2022] DELETE /identity/v3/projects/a9a4ac71270f4ae5bf46a96b837bc82a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:05.332864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.333371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.333371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.353999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:05.355094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:05.412177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 850/1703] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:05.420043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.420412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.420687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.441620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:05.442846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:05.451719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 854/1704] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:05.455960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.466229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a936dfd-0592-4295-b7d2-0864ebb353d0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.472004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.472645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.472900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.473172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.473834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:05.474138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.475742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.487763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 851/1705] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:05.494633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.509975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999571fc-0fcd-412f-b361-6a2442f40cf5 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.515818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.516528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.517269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.517549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.518520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.520546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.552223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 855/1706] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:05.556874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.567186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85835-0f45-4291-962a-fa080f4369c6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.623770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:05.635905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.636677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.636998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.637663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.638656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.640729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.665094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 852/1707] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:05.669124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.679796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fc3bc-1507-4587-99ac-164217242c97 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.684644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.685256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.685534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.685802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.686439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.688053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.698505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 856/1708] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:05.702895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.716878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34b7ca-d22d-4dfa-843a-30e956313f3e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.723509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.725239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.725626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.726086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.726850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.728678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca092e3-1243-4910-a903-c9b656a900fa admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.757924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 853/1709] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:05.763257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.774660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e550d-7126-4fce-9921-99f09b6f2aca admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:05.790540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:05.791892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.791892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.792093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] PATH_INFO: `/v3/projects/ca56b17d74084994a8ac0548ed6731c8/users/b6b190dd1b2b4bf1bab6c830f69b83a6/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.811082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca56b17d74084994a8ac0548ed6731c8, user_id=b6b190dd1b2b4bf1bab6c830f69b83a6, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:05.814219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:05.844371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 857/1710] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:05 2022] PUT /identity/v3/projects/ca56b17d74084994a8ac0548ed6731c8/users/b6b190dd1b2b4bf1bab6c830f69b83a6/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:05.846606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:05.846957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:05.847256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:05.879730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] MFA Rules not processed for user `b6b190dd1b2b4bf1bab6c830f69b83a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:05.880899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:05.955148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 854/1711] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:05.959929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:05.975044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:06.032777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f216a-fa22-410f-99ab-c98ddcf07daa admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.038719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.039349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.039816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.040344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.041310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.043341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.072067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 858/1712] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:05 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:06.081672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.096686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43bed9-16fa-4c21-8ada-d3692b550742 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.159381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.159907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.160224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.160492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.161175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.163252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.189113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 855/1713] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:06 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:06.193150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.204233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fef645-6344-422e-98e4-d65a33b181ce admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.209900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.210524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.210799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.211065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.211710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.213362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.224583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 859/1714] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:06.229180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.239609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eb21d-4d2a-4e6a-bcbc-b7711669907a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.245214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.245870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.246164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.246640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] PATH_INFO: `/v3/projects/0136162746a647d78071d3d0117d9e3a/users/1054cfd9478f41c2902d2d381d8bd5ee/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.262612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0136162746a647d78071d3d0117d9e3a, user_id=1054cfd9478f41c2902d2d381d8bd5ee, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.265273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.280906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 856/1715] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:06 2022] PUT /identity/v3/projects/0136162746a647d78071d3d0117d9e3a/users/1054cfd9478f41c2902d2d381d8bd5ee/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:06.285107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.296509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5625af-5f14-475e-a02e-309790435bc3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.355984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.356628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.356907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.357174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.357824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.359426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851d1e-293e-44b7-9a13-b029e0cc1fd2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.371052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 860/1716] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:06.375475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.386157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919525e-338f-47c4-86d1-ee8a0c81f0b7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.391232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.392290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.392569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.392879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] PATH_INFO: `/v3/projects/0136162746a647d78071d3d0117d9e3a/users/1054cfd9478f41c2902d2d381d8bd5ee/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.395005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0136162746a647d78071d3d0117d9e3a, user_id=1054cfd9478f41c2902d2d381d8bd5ee, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.397785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.408987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 857/1717] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:06 2022] PUT /identity/v3/projects/0136162746a647d78071d3d0117d9e3a/users/1054cfd9478f41c2902d2d381d8bd5ee/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:06.420503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.420674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.420980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.452708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] MFA Rules not processed for user `1054cfd9478f41c2902d2d381d8bd5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:06.453934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:06.526053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 861/1718] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:06.538111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:06.539964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:06.555179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:06.601773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902fc4e-54d8-4532-a118-3b6f26311cfd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:06.606251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:06.606863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:06.607132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:06.607523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:06.668064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:06.669836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e3acc-706c-4ff8-8f98-5949140be849 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:06.680010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 858/1719] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:13.129690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:13.131286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.138366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:13.215688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65485f1-bb4b-4b2e-9a88-59f53353b924 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:13.220311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:13.220964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.221237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.221498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] PATH_INFO: `/v3/users/862c22555d5b44f7a0566553fc13ce5c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.222611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=862c22555d5b44f7a0566553fc13ce5c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:13.224605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:13.274870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862c22555d5b44f7a0566553fc13ce5c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.275514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862c22555d5b44f7a0566553fc13ce5c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.287717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862c22555d5b44f7a0566553fc13ce5c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.297460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862c22555d5b44f7a0566553fc13ce5c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.298203 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862c22555d5b44f7a0566553fc13ce5c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.313096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 862/1720] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:13 2022] DELETE /identity/v3/users/862c22555d5b44f7a0566553fc13ce5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:13.317942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.318418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.318527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.337830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:13.338987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.398923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 859/1721] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:13.409047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:13.410705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.470108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c176-1e02-47a8-9107-6937f68bccdb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:13.474161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:13.474767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.475041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.475311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.484520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:13.485927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a622d6-0669-4434-831f-9e39d47f2ee0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:13.497033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 863/1722] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:13.516262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:13.518144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.530167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deb9a4-4a5e-453f-a456-897a2179ce27 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:13.534919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:13.535548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.535822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.536095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] PATH_INFO: `/v3/users/b6b190dd1b2b4bf1bab6c830f69b83a6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.537259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b190dd1b2b4bf1bab6c830f69b83a6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:13.539198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:13.598164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b190dd1b2b4bf1bab6c830f69b83a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.606713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b190dd1b2b4bf1bab6c830f69b83a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.612754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:13.630527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b190dd1b2b4bf1bab6c830f69b83a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.631425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b190dd1b2b4bf1bab6c830f69b83a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.640574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b190dd1b2b4bf1bab6c830f69b83a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:13.649577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 860/1723] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:13 2022] DELETE /identity/v3/users/b6b190dd1b2b4bf1bab6c830f69b83a6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:13.654465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.655696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.656303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.718706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:13.720067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.749755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:13.805965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 864/1724] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:13.816434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:13.818177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.847564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:13.849520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:13.865983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ae9d0-cec7-432f-b2aa-73840b07677c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:13.872442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:13.873332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.873732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.874133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] PATH_INFO: `/v3/projects/8bcd746102674cb9b173c74eb0bc3686` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.875833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bcd746102674cb9b173c74eb0bc3686)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:13.878132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:13.924896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126c83-e08b-4a56-bd62-55baeb971c18 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:13.942900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:13.943277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:13.943620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:13.943970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:13.956801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:13.958254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:13.968703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 861/1725] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:13.995692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bcd746102674cb9b173c74eb0bc3686', 'request_id': 'req-5660d6e3-2bf4-49a4-842e-678a3778be6f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.003839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bcd746102674cb9b173c74eb0bc3686', 'request_id': 'req-5660d6e3-2bf4-49a4-842e-678a3778be6f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.013175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bcd746102674cb9b173c74eb0bc3686', 'request_id': 'req-5660d6e3-2bf4-49a4-842e-678a3778be6f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.025740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] The token cache is being invalidate because project 8bcd746102674cb9b173c74eb0bc3686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:14.026032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.026873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.031815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 865/1726] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:13 2022] DELETE /identity/v3/projects/8bcd746102674cb9b173c74eb0bc3686 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:14.032861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:14.063928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389ba02-abe0-4266-85ea-ad725f7aae32 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:14.132094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:14.133273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:14.133583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:14.133906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] PATH_INFO: `/v3/users/47e7a350f783460abf8c2755e27b1b12` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:14.135438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e7a350f783460abf8c2755e27b1b12)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:14.137883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:14.213883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e7a350f783460abf8c2755e27b1b12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.221786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e7a350f783460abf8c2755e27b1b12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.240177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e7a350f783460abf8c2755e27b1b12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.240950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e7a350f783460abf8c2755e27b1b12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.259543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e7a350f783460abf8c2755e27b1b12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.267893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 862/1727] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:14 2022] DELETE /identity/v3/users/47e7a350f783460abf8c2755e27b1b12 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:14.318090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:14.320205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:14.337970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660d6e3-2bf4-49a4-842e-678a3778be6f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:14.426551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:14.427406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:14.427684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:14.427952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] PATH_INFO: `/v3/projects/ca56b17d74084994a8ac0548ed6731c8` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:14.429169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca56b17d74084994a8ac0548ed6731c8)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:14.431127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:14.483815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca56b17d74084994a8ac0548ed6731c8', 'request_id': 'req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.486080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:14.491026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca56b17d74084994a8ac0548ed6731c8', 'request_id': 'req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.501794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca56b17d74084994a8ac0548ed6731c8', 'request_id': 'req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.538137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] The token cache is being invalidate because project ca56b17d74084994a8ac0548ed6731c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:14.538919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.539686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.541631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 866/1728] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:14 2022] DELETE /identity/v3/projects/ca56b17d74084994a8ac0548ed6731c8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:14.545879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:14.547881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:14.557823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:14.593445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a89062-c140-4c07-b15a-34f1150c8553 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:14.670115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:14.670776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:14.671053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:14.671322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] PATH_INFO: `/v3/users/1054cfd9478f41c2902d2d381d8bd5ee` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:14.672487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1054cfd9478f41c2902d2d381d8bd5ee)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:14.674443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:14.733222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1054cfd9478f41c2902d2d381d8bd5ee'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.742764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1054cfd9478f41c2902d2d381d8bd5ee'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.756720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1054cfd9478f41c2902d2d381d8bd5ee'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.757476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1054cfd9478f41c2902d2d381d8bd5ee'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.769663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1054cfd9478f41c2902d2d381d8bd5ee'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:14.784227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 863/1729] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:14 2022] DELETE /identity/v3/users/1054cfd9478f41c2902d2d381d8bd5ee => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:15.022950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.024541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.044338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcf40b-ff4c-4e88-9b9f-2119feea945a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.124428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.125030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.147808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.148637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.148912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.149182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] PATH_INFO: `/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.161541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244cdc1f16fb4aeb9bf1d287aa7c6e80)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.164583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.272225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c81ea-1cee-4bdb-b925-ed885524cf53 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.285679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244cdc1f16fb4aeb9bf1d287aa7c6e80', 'request_id': 'req-b547c446-adf9-4c27-be4e-070df36e3758'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.294497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244cdc1f16fb4aeb9bf1d287aa7c6e80', 'request_id': 'req-b547c446-adf9-4c27-be4e-070df36e3758'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.302711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:15.304102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244cdc1f16fb4aeb9bf1d287aa7c6e80', 'request_id': 'req-b547c446-adf9-4c27-be4e-070df36e3758'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.324082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] The token cache is being invalidate because project 244cdc1f16fb4aeb9bf1d287aa7c6e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:15.324729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.325036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.326541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 867/1730] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:15 2022] DELETE /identity/v3/projects/244cdc1f16fb4aeb9bf1d287aa7c6e80 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:15.329680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.331025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.344262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.344905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.345328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.345598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] PATH_INFO: `/v3/projects/0136162746a647d78071d3d0117d9e3a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.346719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0136162746a647d78071d3d0117d9e3a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.348932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.395773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0136162746a647d78071d3d0117d9e3a', 'request_id': 'req-7516513b-6b65-463b-9605-0d9288e860b8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.399202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c446-adf9-4c27-be4e-070df36e3758 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.402475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0136162746a647d78071d3d0117d9e3a', 'request_id': 'req-7516513b-6b65-463b-9605-0d9288e860b8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.410196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0136162746a647d78071d3d0117d9e3a', 'request_id': 'req-7516513b-6b65-463b-9605-0d9288e860b8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.412348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:15.421501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] The token cache is being invalidate because project 0136162746a647d78071d3d0117d9e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:15.421855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.422619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7516513b-6b65-463b-9605-0d9288e860b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.424167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 864/1731] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:15 2022] DELETE /identity/v3/projects/0136162746a647d78071d3d0117d9e3a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:15.425388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9600e-55e4-4736-b0ad-4fe4bc5caf46 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.425667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9600e-55e4-4736-b0ad-4fe4bc5caf46 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.425934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9600e-55e4-4736-b0ad-4fe4bc5caf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.445566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-95e9600e-55e4-4736-b0ad-4fe4bc5caf46 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:15.446771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-95e9600e-55e4-4736-b0ad-4fe4bc5caf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.467171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.467818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.468147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.468431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] PATH_INFO: `/v3/users/34aff971c9ac4db397b1dd3a3833f485` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.469605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=34aff971c9ac4db397b1dd3a3833f485)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.471615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.511488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 865/1732] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:15.512864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33dfcb-1bf9-4c69-bc71-3f1bf13bdd51 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.513136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33dfcb-1bf9-4c69-bc71-3f1bf13bdd51 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.513405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33dfcb-1bf9-4c69-bc71-3f1bf13bdd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.531756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34aff971c9ac4db397b1dd3a3833f485'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.532504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34aff971c9ac4db397b1dd3a3833f485'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.534566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8e33dfcb-1bf9-4c69-bc71-3f1bf13bdd51 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:15.536513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8e33dfcb-1bf9-4c69-bc71-3f1bf13bdd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.551006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34aff971c9ac4db397b1dd3a3833f485'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.560493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 866/1733] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:15.561102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34aff971c9ac4db397b1dd3a3833f485'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.561852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-acf483c0-f023-41a5-9851-189760d321ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34aff971c9ac4db397b1dd3a3833f485'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:15.569696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2236d9-3b35-4220-8ac7-6a7e90eb4bce None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.570285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2236d9-3b35-4220-8ac7-6a7e90eb4bce None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.570852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2236d9-3b35-4220-8ac7-6a7e90eb4bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.576619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 868/1734] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:15 2022] DELETE /identity/v3/users/34aff971c9ac4db397b1dd3a3833f485 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:15.580744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.581106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.581447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.595716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9f2236d9-3b35-4220-8ac7-6a7e90eb4bce None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:15.596950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9f2236d9-3b35-4220-8ac7-6a7e90eb4bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.601066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:15.602232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.645120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:15.660016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 867/1735] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:15.661410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 869/1736] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:15.661542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.661542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.661834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.664259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.673867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72f87b-ad15-43cf-96b3-e10504dbf672 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.678710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.679350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.679741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.680010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.680692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:15.680989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.681093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:15.682255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.682578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.690166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 868/1737] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:15.690951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.700712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7039be-0ae0-470d-861c-91bef03e835c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.701453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 870/1738] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:29:15.705507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.705826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.706439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.706720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.707041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.707442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:15.707696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:15.707996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.716219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.723015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 869/1739] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:15.723866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.737556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddd9cd-eb01-4d12-b8bf-cb92dab2b8d8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.742700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.743304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.743578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.743843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.743971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:15.744629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.746794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.776051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ac5a8-859c-460b-aeb6-1e78554a0756 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.777540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 870/1740] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:15.778341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.829036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.829676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.829952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.830219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.839143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.839439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4caef-ee71-4dbb-9330-df14e980f24b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.840575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.844232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.844758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.845035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.845332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.845983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.847913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.851672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 871/1741] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:15.852583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.862269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e445ed0-7354-4977-88d1-2d860ca5853c service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.867195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.867888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.868260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.868564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.869336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.871541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.880883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 871/1742] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:15.883139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.897422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 872/1743] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:15.901558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:15.913260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043cd7-a083-495c-8f9a-7f6a8965f4db admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.965034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82122b-f8df-4675-bf9b-cadee9f4b4b6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:15.971661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.972480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.972855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.973273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.974358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.976674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:15.990534 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:15.991198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:15.991476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:15.991886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:15.992600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:15.994238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.005656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 872/1744] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:15 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:16.006427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 873/1745] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:16.009230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.011962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.021465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652fe7b-7616-4b10-a5fb-e4500396a361 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.023479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ece2d2-301d-4b0c-ba1c-a59b5085dd48 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.027928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.029066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.029405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.029690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.029957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.030211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.030592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.031117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.031554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.032240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.032700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.035260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.051935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 874/1746] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:16.056358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 873/1747] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:16.057162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.057651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.069003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d46d4-a53c-4ef5-97c8-02d4002fec51 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.071833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53cb24-9621-4cb6-bcd2-a976b7cc52c6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.074271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.074508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.074984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.075261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.075909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.077553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.078391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.079056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.079330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.079597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] PATH_INFO: `/v3/projects/fda7612654a6480b8eeec660bc7260ff/users/f226b031d1874bc79bec2188796ac8d0/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.089866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 874/1748] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:16.093872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.098603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda7612654a6480b8eeec660bc7260ff, user_id=f226b031d1874bc79bec2188796ac8d0, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.100805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:16.101012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067aef8-05f9-4f67-a003-dffa190761a5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.105166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afe1b5-ad0f-4a09-8047-95549228afd3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.112688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.113647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.114641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.115064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] PATH_INFO: `/v3/projects/05aa2418cebe4625b97d8ef5baf70a35/users/b1146abe4f9042968fd32a639cd504bc/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.129459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 875/1749] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:16 2022] PUT /identity/v3/projects/fda7612654a6480b8eeec660bc7260ff/users/f226b031d1874bc79bec2188796ac8d0/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:16.132622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479bcc-f7c6-4325-830b-0e3aa635bc96 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.132912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479bcc-f7c6-4325-830b-0e3aa635bc96 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.134900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479bcc-f7c6-4325-830b-0e3aa635bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.141074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05aa2418cebe4625b97d8ef5baf70a35, user_id=b1146abe4f9042968fd32a639cd504bc, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.144017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.162476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 875/1750] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:16 2022] PUT /identity/v3/projects/05aa2418cebe4625b97d8ef5baf70a35/users/b1146abe4f9042968fd32a639cd504bc/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:16.167047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.168868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.174108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d9479bcc-f7c6-4325-830b-0e3aa635bc96 None None] MFA Rules not processed for user `f226b031d1874bc79bec2188796ac8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:16.175545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d9479bcc-f7c6-4325-830b-0e3aa635bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.190187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:16.252844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 876/1751] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:16.254123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.254428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.254693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.265897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2fc78-9c0f-4ea9-a28f-6a092c50a605 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.271237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.271962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.272330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.272658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] PATH_INFO: `/v3/projects/3715866a8e3b448284724cea72a3055e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.273917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3715866a8e3b448284724cea72a3055e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.275029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:16.275940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.276601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.283984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 877/1752] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:16.284793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.304345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298048f1-cd90-49e9-b917-698ce264f0a8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.310248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.311118 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.311396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.311662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.312357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.314449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.334767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3715866a8e3b448284724cea72a3055e', 'request_id': 'req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.344655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3715866a8e3b448284724cea72a3055e', 'request_id': 'req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.355316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3715866a8e3b448284724cea72a3055e', 'request_id': 'req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.357946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 878/1753] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:16 2022] POST /identity/v3/projects => generated 393 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:16.359075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.361166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.370545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] The token cache is being invalidate because project 3715866a8e3b448284724cea72a3055e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:16.370925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.376047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.378161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 876/1754] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:16 2022] DELETE /identity/v3/projects/3715866a8e3b448284724cea72a3055e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:16.380160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.380589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.471887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035f64f-f95a-40b9-9f14-e7cde2540e11 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.477447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.477724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e62e-bd44-4cc2-a32f-9a0c16fedec5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.478251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.478628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.479267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.482873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.514236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:16.566430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.573585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.584677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 879/1755] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:16.585570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.587725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.588908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.589417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.590002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.590911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.591410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.594971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.606857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df1233-67f2-417d-8b2a-fa75de38df21 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.613104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.613795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.614068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.614346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] PATH_INFO: `/v3/users/5ce24c4d355d4305b3f1d206f9fa50bf` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.615553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce24c4d355d4305b3f1d206f9fa50bf)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.618476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.643670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 877/1756] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:16 2022] POST /identity/v3/users => generated 303 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:16.645631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.661658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646ab7b-ff20-4c2d-b127-4a9a90d60a5d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.667151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.667827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.668132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.668475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.669172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.671719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.678484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:16.690854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 878/1757] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:16.694878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.707334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275cc7f-e959-477f-bd2e-ecdac1566c05 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.710220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce24c4d355d4305b3f1d206f9fa50bf'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.711150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce24c4d355d4305b3f1d206f9fa50bf'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.712521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.713573 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.713872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.714694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] PATH_INFO: `/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d/users/fdece98b713c4890b06c3cca926480db/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.729455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce24c4d355d4305b3f1d206f9fa50bf'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.733061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05c551d7dbc4cd8ac36b3d615b8f71d, user_id=fdece98b713c4890b06c3cca926480db, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.736801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.738357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce24c4d355d4305b3f1d206f9fa50bf'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.739246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce24c4d355d4305b3f1d206f9fa50bf'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:16.752085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 879/1758] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:16 2022] PUT /identity/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d/users/fdece98b713c4890b06c3cca926480db/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:16.756075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.756577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 880/1759] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:16 2022] DELETE /identity/v3/users/5ce24c4d355d4305b3f1d206f9fa50bf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:16.760080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:16.849961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb5cfb-4092-47ce-81e3-553a4895dfd9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.933264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.933264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.933264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.933264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.934246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.937050 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fee95-be0a-4110-93bf-0f0690448c41 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:16.955915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 880/1760] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:16.957162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:16.971540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661025-5ebc-4a62-bf8e-fa5214856245 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:16.977595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:16.978463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:16.979087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:16.979354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] PATH_INFO: `/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d/users/fdece98b713c4890b06c3cca926480db/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:16.981520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05c551d7dbc4cd8ac36b3d615b8f71d, user_id=fdece98b713c4890b06c3cca926480db, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:16.984918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.000158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 881/1761] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:16 2022] PUT /identity/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d/users/fdece98b713c4890b06c3cca926480db/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:17.009169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.010328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.010328 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.066535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] MFA Rules not processed for user `fdece98b713c4890b06c3cca926480db`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:17.067836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.072289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:17.084888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.088541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.162536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 881/1762] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:17.174222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.175960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.187144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb91d0-c0eb-43db-8b8d-b88da1f916d7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.193051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.193805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.194149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.194500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] PATH_INFO: `/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.195955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28fd7b22d1d4188a3aaed9b29ea4758)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.197997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.248168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f6e8f-605b-46d2-8379-ee2947fb0c99 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.273571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28fd7b22d1d4188a3aaed9b29ea4758', 'request_id': 'req-e260a9dc-55c7-493e-83a7-ac979a92bdac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:17.284062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28fd7b22d1d4188a3aaed9b29ea4758', 'request_id': 'req-e260a9dc-55c7-493e-83a7-ac979a92bdac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:17.296159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28fd7b22d1d4188a3aaed9b29ea4758', 'request_id': 'req-e260a9dc-55c7-493e-83a7-ac979a92bdac'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:17.320493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] The token cache is being invalidate because project b28fd7b22d1d4188a3aaed9b29ea4758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:17.320974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:17.323563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e260a9dc-55c7-493e-83a7-ac979a92bdac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:17.333758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 882/1763] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:17 2022] DELETE /identity/v3/projects/b28fd7b22d1d4188a3aaed9b29ea4758 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:17.335384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e646-9894-4d17-8433-844f74422c66 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.335670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e646-9894-4d17-8433-844f74422c66 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.335939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e646-9894-4d17-8433-844f74422c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.356188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.358049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.358797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.359086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.365887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.380681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bc23e646-9894-4d17-8433-844f74422c66 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:17.382301 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bc23e646-9894-4d17-8433-844f74422c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.426768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:17.487422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.490111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.504661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 883/1764] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:17.507091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.507444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.507921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.509060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 882/1765] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:17.536516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:17.538205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.552153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 884/1766] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:17.553868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.569610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.570026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2444840-e5b6-4702-b3a5-984b7636ee55 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.571052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.575358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.575898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.576199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.576469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.577139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:17.577435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.579036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.580923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:17.589754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0b0fb-9c66-43ba-9bfa-d7f2bea06950 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.592680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.593455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.593755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.593997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.596404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.598068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 883/1767] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:29:17.601891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.615502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d115fd-01e5-4b42-9b1f-9ecf7bf74bd7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.620570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.621194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.621612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.622060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.622754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.624800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.662640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 884/1768] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:17.666837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.679548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.681188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f458fd-2839-4510-bfe1-55c2759f9f2c service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.691999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 885/1769] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:17.742979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.743646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.744141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.751836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9369b3-fc0e-46b5-b653-b63bd4ddb693 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.756726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.757342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.757612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.757878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.759529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.761527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.804553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 885/1770] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:17.810545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.817086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] MFA Rules not processed for user `b1146abe4f9042968fd32a639cd504bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:17.820203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818a180-e18f-4212-a10a-b5b19851850b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.821274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:17.825474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.826101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.826383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.826656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.827307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.828944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.841030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 886/1771] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:17.843786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.855545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3d081-e47c-4e14-9283-dd509a59a4e2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.860598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.861279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.861630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.861969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.862778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.864488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.876903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 887/1772] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:17.881428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.894319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7de4-fa62-4a9f-bf07-0a3df0da647f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:17.901735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:17.902825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.903255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.904064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] PATH_INFO: `/v3/projects/275bd03a752e46f6bab988b763d9a1b4/users/662fbcac1c1c4bdd8c441e4bdff8aa76/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.924133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275bd03a752e46f6bab988b763d9a1b4, user_id=662fbcac1c1c4bdd8c441e4bdff8aa76, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:17.926697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77a893-3571-48cc-b6c9-6b0400d359d5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:17.958739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 888/1773] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:17 2022] PUT /identity/v3/projects/275bd03a752e46f6bab988b763d9a1b4/users/662fbcac1c1c4bdd8c441e4bdff8aa76/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:17.960950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:17.961419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:17.962429 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:17.982685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 886/1774] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:17.997461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:17.999991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:18.003181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] MFA Rules not processed for user `662fbcac1c1c4bdd8c441e4bdff8aa76`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:18.005815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:18.071864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:18.075846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a43ee-7ffd-4c59-9be7-ddc537b1ba7f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.090503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 889/1775] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:18.096716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.103920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:18.148009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.148009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.148009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.148009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.207533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2f09c-7ce5-46a5-b4ea-885e71ca4a6b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.212631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.213396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.213717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.214089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.214782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.216841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.229724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.231173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.240299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 887/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:18.245020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 890/1777] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:18 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:18.252272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.263236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51f3d-4693-470c-b51d-d1552aecc543 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.321006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.321622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.321948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.322159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.322787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.324825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.349499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 888/1778] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:18 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:18.350811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.361126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada532-6ff8-4f7b-baa7-eb53d78feb05 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.366720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.368169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.368502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.368863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.369982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.371957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.384159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 891/1779] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:18.388845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.399891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6674d-04c7-488a-859c-3149f86d17ad admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.404683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.405349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.405629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.405902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] PATH_INFO: `/v3/projects/baa7d040023343a6931989784d8837a9/users/e8f29b442ecc4f1cbe0dc1ae638f8692/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.421598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa7d040023343a6931989784d8837a9, user_id=e8f29b442ecc4f1cbe0dc1ae638f8692, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.424333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.437926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 889/1780] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:18 2022] PUT /identity/v3/projects/baa7d040023343a6931989784d8837a9/users/e8f29b442ecc4f1cbe0dc1ae638f8692/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:18.442000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.453062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9550e-4d4f-4ca0-9f12-c0e61af4d155 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.536763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.536763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.536763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.536763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.537244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.538571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14c6b5-17fa-4705-a19c-e4778fd17adb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.555976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 892/1781] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:18.561520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.572521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6951-c682-4d7d-9b08-f61058bafd54 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.578253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.578959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.579259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.579540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] PATH_INFO: `/v3/projects/baa7d040023343a6931989784d8837a9/users/e8f29b442ecc4f1cbe0dc1ae638f8692/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.581571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa7d040023343a6931989784d8837a9, user_id=e8f29b442ecc4f1cbe0dc1ae638f8692, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.584360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.595792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 890/1782] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:18 2022] PUT /identity/v3/projects/baa7d040023343a6931989784d8837a9/users/e8f29b442ecc4f1cbe0dc1ae638f8692/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:18.606760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.607261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.607360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.636242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] MFA Rules not processed for user `e8f29b442ecc4f1cbe0dc1ae638f8692`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:18.637440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:18.704562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:18.708617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 893/1783] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:18.720277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:18.735717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:18.781113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726c3b2-4de0-423c-8023-6ca3a419bd0e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:18.785264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:18.785887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:18.786167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:18.786452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:18.796499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:18.798006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0ff00-ae99-4aa0-8226-d88663ed025e service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:18.807772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 891/1784] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:20.210295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:20.283412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da38a17-493a-4573-b03b-97a897047e5a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:20.294843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:20.295495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:20.295770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:20.296037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] PATH_INFO: `/v3/users/662fbcac1c1c4bdd8c441e4bdff8aa76` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:20.297248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662fbcac1c1c4bdd8c441e4bdff8aa76)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:20.299199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:20.347198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662fbcac1c1c4bdd8c441e4bdff8aa76'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:20.354420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662fbcac1c1c4bdd8c441e4bdff8aa76'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:20.368065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662fbcac1c1c4bdd8c441e4bdff8aa76'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:20.369277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662fbcac1c1c4bdd8c441e4bdff8aa76'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:20.377217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662fbcac1c1c4bdd8c441e4bdff8aa76'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:20.383077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 894/1785] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:20 2022] DELETE /identity/v3/users/662fbcac1c1c4bdd8c441e4bdff8aa76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:20.387854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9cf2-ebd8-4241-9ab4-7df023928716 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:20.388378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9cf2-ebd8-4241-9ab4-7df023928716 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:20.388602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd9cf2-ebd8-4241-9ab4-7df023928716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:20.410405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-38dd9cf2-ebd8-4241-9ab4-7df023928716 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:20.411728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-38dd9cf2-ebd8-4241-9ab4-7df023928716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:20.473251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 892/1786] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:20.483014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:20.484638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:20.543345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb829-119e-4103-ad55-1636c3a09962 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:20.547278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:20.547892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:20.548196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:20.548488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:20.558121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:20.559192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:20.569858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 895/1787] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:20.721863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:20.722153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:20.722484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:20.743140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] MFA Rules not processed for user `b1146abe4f9042968fd32a639cd504bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:20.745488 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:20.806812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 893/1788] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:20.904210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:20.906991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:20.920607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4351b50-6970-4b6b-b6d3-67492683c647 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:20.925989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:20.926715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:20.927051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:20.927434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] PATH_INFO: `/v3/projects/275bd03a752e46f6bab988b763d9a1b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:20.928636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=275bd03a752e46f6bab988b763d9a1b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:20.930736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:20.950221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:20.997160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275bd03a752e46f6bab988b763d9a1b4', 'request_id': 'req-0ea6f992-61b7-477c-bdfd-96402582fc24'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.009547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275bd03a752e46f6bab988b763d9a1b4', 'request_id': 'req-0ea6f992-61b7-477c-bdfd-96402582fc24'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.028290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275bd03a752e46f6bab988b763d9a1b4', 'request_id': 'req-0ea6f992-61b7-477c-bdfd-96402582fc24'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.043072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] The token cache is being invalidate because project 275bd03a752e46f6bab988b763d9a1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:21.043374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.044058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.046100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 896/1789] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:20 2022] DELETE /identity/v3/projects/275bd03a752e46f6bab988b763d9a1b4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:21.049780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:21.062748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa031d-961b-4990-95c4-879b4b33f249 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:21.150187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:21.151347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:21.151784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:21.152047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] PATH_INFO: `/v3/users/e8f29b442ecc4f1cbe0dc1ae638f8692` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:21.153300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f29b442ecc4f1cbe0dc1ae638f8692)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:21.155200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:21.157370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:21.211113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f29b442ecc4f1cbe0dc1ae638f8692'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.211763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f29b442ecc4f1cbe0dc1ae638f8692'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.230780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f29b442ecc4f1cbe0dc1ae638f8692'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.240699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f29b442ecc4f1cbe0dc1ae638f8692'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.241640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d1aeee71-d04c-4eac-b043-2083deea1856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f29b442ecc4f1cbe0dc1ae638f8692'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.259248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 894/1790] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:21 2022] DELETE /identity/v3/users/e8f29b442ecc4f1cbe0dc1ae638f8692 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:21.735811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:21.739401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:21.755566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f992-61b7-477c-bdfd-96402582fc24 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:21.817915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:21.818618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:21.818921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:21.819217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] PATH_INFO: `/v3/projects/baa7d040023343a6931989784d8837a9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:21.820548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa7d040023343a6931989784d8837a9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:21.822915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:21.873146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa7d040023343a6931989784d8837a9', 'request_id': 'req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.879687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa7d040023343a6931989784d8837a9', 'request_id': 'req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.889769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa7d040023343a6931989784d8837a9', 'request_id': 'req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.901180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] The token cache is being invalidate because project baa7d040023343a6931989784d8837a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:21.901599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.902235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-13161b35-3f93-4879-a4ff-cc8bc8e4cf1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:21.903735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 897/1791] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:21 2022] DELETE /identity/v3/projects/baa7d040023343a6931989784d8837a9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:21.908696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:21.909162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:21.909285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:21.929535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:21.930801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:21.996000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 895/1792] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:22.005127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.005409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.005683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.026014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:22.027391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.035621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 898/1793] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:22.039876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.050517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452cd25-12de-476e-b202-6a68cc74c18a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.056412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.057100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.057375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.057673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.058348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:22.058645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.060433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.076065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 896/1794] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:22.077270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.087620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6d37-212f-4186-a8b4-aca4fcfc9bb9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.093802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.094418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.094698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.095082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.095849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.098058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.127516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 899/1795] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:22.131715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.142185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8212-563b-46cc-92de-5cbeca6930ca admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.204972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.205712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.206377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.206857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.207691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.209669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.221803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:22.237095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 897/1796] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:22.238408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.248771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7764a8a-0950-495d-a801-82929b7f080e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.253700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.254323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.254602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.254875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.255674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.257472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.258861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:22.270321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 900/1797] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:22.274124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.286209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276727-29b0-4009-995d-0a950b0e86a0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.291921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.292722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.293142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.293490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.294831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.295986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf386e6-7df9-45c0-844c-1bfcf6771cb0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.311777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 898/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:22.313216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.324955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f197-2480-4753-8f46-1b59a4f7e4ec admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.329877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.330558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.330838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.331111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] PATH_INFO: `/v3/projects/58faac939b704126a61c79cebd4eb8b3/users/22d0b942274d4ab9b965c0b8c559a527/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.347377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58faac939b704126a61c79cebd4eb8b3, user_id=22d0b942274d4ab9b965c0b8c559a527, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.350192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.365829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 901/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:22 2022] PUT /identity/v3/projects/58faac939b704126a61c79cebd4eb8b3/users/22d0b942274d4ab9b965c0b8c559a527/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:22.376507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.376801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.377115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.415593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] MFA Rules not processed for user `22d0b942274d4ab9b965c0b8c559a527`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:22.417566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.489225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 899/1800] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:22.490563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.492284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.494287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.557308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af743a16-5382-4aca-8f0e-5ae3873823f9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.561998 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.562623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.562896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.563162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.563451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62bd52-49c5-436d-9a6c-037797115b1a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.563796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.565695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.574989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.574989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.574989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.575301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] PATH_INFO: `/v3/users/f226b031d1874bc79bec2188796ac8d0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.576453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f226b031d1874bc79bec2188796ac8d0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.578626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.622163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 900/1801] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:22.623526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.631012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f226b031d1874bc79bec2188796ac8d0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:22.638411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f226b031d1874bc79bec2188796ac8d0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:22.654396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f226b031d1874bc79bec2188796ac8d0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:22.655826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f226b031d1874bc79bec2188796ac8d0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:22.665399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ecd91d0d-df0d-4515-8c1a-6cb1dc93f197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f226b031d1874bc79bec2188796ac8d0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:22.672036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 902/1802] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:22 2022] DELETE /identity/v3/users/f226b031d1874bc79bec2188796ac8d0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:22.676499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.676780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.677053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.696877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b0b0-3f26-486b-874c-7628957ae990 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.702194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:22.702702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:22.704351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.774068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.775254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.775593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.775928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.776675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.778616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.801677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:22.804206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 901/1803] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:22.808860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.814003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.827642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff728-972e-40fa-b761-4bcd3ecac768 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.830156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 903/1804] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:22.832489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.833142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.833503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.833782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.834436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.836074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.839841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.841198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:22.847108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 902/1805] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:22.851030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.861688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47928042-ef53-4876-8882-b320bb5cb38d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.866273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.866917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.867194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.867466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] PATH_INFO: `/v3/projects/e6de5aaa05dc493792f5fa34f31cd205/users/f5d9f28aefbb4bc29182c9857dd42826/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.884216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6de5aaa05dc493792f5fa34f31cd205, user_id=f5d9f28aefbb4bc29182c9857dd42826, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.886934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:22.898308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c723b8-efef-4c1f-a2c9-e9dba7bdc8e5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.900041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 903/1806] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:22 2022] PUT /identity/v3/projects/e6de5aaa05dc493792f5fa34f31cd205/users/f5d9f28aefbb4bc29182c9857dd42826/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:22.904482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:22.948890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.949503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.949801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.950063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.965653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828007-c482-4401-b1f5-8f02e0b39bd8 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:22.976818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:22.977970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:22.978454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:22.978890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:22.979757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:22.981605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.003566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 904/1807] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:23.005297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.015945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012617e-2204-461d-9733-710aa9f8fdac admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.021253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.022077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.022358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.022624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] PATH_INFO: `/v3/projects/e6de5aaa05dc493792f5fa34f31cd205/users/f5d9f28aefbb4bc29182c9857dd42826/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.024546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6de5aaa05dc493792f5fa34f31cd205, user_id=f5d9f28aefbb4bc29182c9857dd42826, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.026692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.027291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d92dd-0fb1-4de7-ba9d-f39d836124e5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.028230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.037602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 905/1808] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:23 2022] PUT /identity/v3/projects/e6de5aaa05dc493792f5fa34f31cd205/users/f5d9f28aefbb4bc29182c9857dd42826/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:23.038233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 904/1809] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:23.048535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.048815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.049086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.084304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] MFA Rules not processed for user `f5d9f28aefbb4bc29182c9857dd42826`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:23.085367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.124987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:23.202177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 906/1810] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:23.210384 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.211882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.290266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830059d3-1f24-448d-acaf-5d45e8f8182f service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.301052 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:23.350920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.351550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.351826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.352095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.354985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.356723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.362287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.363750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.384770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 905/1811] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:23.418872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.453110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68621fb4-f3db-4304-ab95-5deda3b47063 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.458501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.459157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.459460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.459726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] PATH_INFO: `/v3/projects/fda7612654a6480b8eeec660bc7260ff` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.460878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda7612654a6480b8eeec660bc7260ff)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.463342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.491049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dd997-5d7f-4b5e-9432-10c494ff9628 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.495865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.496619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.496954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.497290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] PATH_INFO: `/v3/users/22d0b942274d4ab9b965c0b8c559a527` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.498610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d0b942274d4ab9b965c0b8c559a527)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.500799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.523252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda7612654a6480b8eeec660bc7260ff', 'request_id': 'req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.532018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda7612654a6480b8eeec660bc7260ff', 'request_id': 'req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.544106 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda7612654a6480b8eeec660bc7260ff', 'request_id': 'req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.556403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d0b942274d4ab9b965c0b8c559a527'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.557870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] The token cache is being invalidate because project fda7612654a6480b8eeec660bc7260ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:23.558182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.558875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.560587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 907/1812] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:23 2022] DELETE /identity/v3/projects/fda7612654a6480b8eeec660bc7260ff => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:23.564721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.566140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.566635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d0b942274d4ab9b965c0b8c559a527'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.580664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d0b942274d4ab9b965c0b8c559a527'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.581421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d0b942274d4ab9b965c0b8c559a527'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.595343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-511602c2-2618-40e6-8c26-08aec2186a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d0b942274d4ab9b965c0b8c559a527'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.602833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 906/1813] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:23 2022] DELETE /identity/v3/users/22d0b942274d4ab9b965c0b8c559a527 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:23.607366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.607657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.607987 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.632556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:23.633693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.651767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b7b28-d30a-41ab-a3b9-78bcdf3cd6c5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.656235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:23.669615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:23.711731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 907/1814] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:23.720372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.721058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.721340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.721608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] PATH_INFO: `/v3/users/fdece98b713c4890b06c3cca926480db` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.722872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdece98b713c4890b06c3cca926480db)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.724818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.725449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:23.726823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.787394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdece98b713c4890b06c3cca926480db'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.788279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdece98b713c4890b06c3cca926480db'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.808066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be5fe0-370d-4323-9b35-437a0517522e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:23.811979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:23.812657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:23.813003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:23.813366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:23.815598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdece98b713c4890b06c3cca926480db'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.816653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:23.828811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdece98b713c4890b06c3cca926480db'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.829530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdece98b713c4890b06c3cca926480db'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:23.835683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:23.837868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:23.848164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 908/1815] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:23 2022] DELETE /identity/v3/users/fdece98b713c4890b06c3cca926480db => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:23.854655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 908/1816] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:24.495865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:24.497818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.519294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:24.520912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.534736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849ae23-b74e-4d2a-a935-77abd420fd9e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:24.623774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea7ff1-e6c4-4362-9827-e54b2498ea6e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:24.627002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:24.627680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.627981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.628276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] PATH_INFO: `/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.629441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:24.629571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05c551d7dbc4cd8ac36b3d615b8f71d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:24.630786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.631219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.631526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:24.631789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] PATH_INFO: `/v3/projects/58faac939b704126a61c79cebd4eb8b3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.633222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58faac939b704126a61c79cebd4eb8b3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:24.635219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:24.671596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:24.685405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05c551d7dbc4cd8ac36b3d615b8f71d', 'request_id': 'req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.692057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05c551d7dbc4cd8ac36b3d615b8f71d', 'request_id': 'req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.693692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58faac939b704126a61c79cebd4eb8b3', 'request_id': 'req-7be53c27-8590-4539-bdeb-c0b5c6b5803f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.700981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58faac939b704126a61c79cebd4eb8b3', 'request_id': 'req-7be53c27-8590-4539-bdeb-c0b5c6b5803f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.701739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05c551d7dbc4cd8ac36b3d615b8f71d', 'request_id': 'req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.708739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58faac939b704126a61c79cebd4eb8b3', 'request_id': 'req-7be53c27-8590-4539-bdeb-c0b5c6b5803f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.713353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] The token cache is being invalidate because project b05c551d7dbc4cd8ac36b3d615b8f71d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:24.713723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.714468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e5a42c97-dd33-4f1a-97d3-0c760ae5504b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.716549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 909/1817] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:24 2022] DELETE /identity/v3/projects/b05c551d7dbc4cd8ac36b3d615b8f71d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:24.721219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5cfc9-379a-46b0-9b3c-9614b7934815 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.721954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5cfc9-379a-46b0-9b3c-9614b7934815 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.721954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5cfc9-379a-46b0-9b3c-9614b7934815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.724434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] The token cache is being invalidate because project 58faac939b704126a61c79cebd4eb8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:24.724789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.725688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.727483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 909/1818] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:24 2022] DELETE /identity/v3/projects/58faac939b704126a61c79cebd4eb8b3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:24.731338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:24.732737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.745728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-70b5cfc9-379a-46b0-9b3c-9614b7934815 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:24.747115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-70b5cfc9-379a-46b0-9b3c-9614b7934815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.752381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:24.831627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be53c27-8590-4539-bdeb-c0b5c6b5803f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:24.837798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:24.838446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.838848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.839127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] PATH_INFO: `/v3/users/f5d9f28aefbb4bc29182c9857dd42826` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.840517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d9f28aefbb4bc29182c9857dd42826)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:24.842458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:24.843320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 910/1819] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:24.851857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.852400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.852684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.873291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:24.874497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.884041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 911/1820] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:24.889080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:24.899466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6563d2-94f7-4a17-b27b-6785c5245115 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:24.907110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d9f28aefbb4bc29182c9857dd42826'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.908019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:24.908579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d9f28aefbb4bc29182c9857dd42826'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.909755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.910034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.910303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.910964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:24.911266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:24.913063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:24.926936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 912/1821] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:29:24.927087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d9f28aefbb4bc29182c9857dd42826'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.930805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:24.932941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:24.936420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d9f28aefbb4bc29182c9857dd42826'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.937449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d9f28aefbb4bc29182c9857dd42826'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:24.952873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a228f9-4824-4d3f-a2ba-94f045a840b6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:24.952873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:24.952873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:24.952873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:24.953311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 910/1822] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:24 2022] DELETE /identity/v3/users/f5d9f28aefbb4bc29182c9857dd42826 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:24.953398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:24.953478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:24.962914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:24.993747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 913/1823] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:24 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:24.997950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.047623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:25.080652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f5e5-7713-48f6-90df-ce8a81dcd945 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.148849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.149469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.149740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.150002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.150635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.152721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.195538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 911/1824] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:24 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:25.196563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.208141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42911f63-dc06-4dce-a2fb-d678b6b89269 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.212717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.213528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.213949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.214498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.214826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.216506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.230557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 914/1825] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:25.232551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.244285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630c23c-5dda-4d8d-9926-5eea57119816 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.249503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.250121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.250390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.250660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.251298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.252935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.268305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 912/1826] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:25.268508 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.269012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:25.271214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.291142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae489-9401-42c4-81b8-2f523878b0b1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.291550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37eef-3742-42da-8e46-0f73ed69f9d0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.296161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.296767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.297047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.297321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] PATH_INFO: `/v3/projects/e6de5aaa05dc493792f5fa34f31cd205` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.298511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6de5aaa05dc493792f5fa34f31cd205)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.300580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.307322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.308078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.308393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.308663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] PATH_INFO: `/v3/projects/1d538e48e3c4489eaa9ccf5325877992/users/acd7b417776a4576bebeb7ae6edcccaa/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.325246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d538e48e3c4489eaa9ccf5325877992, user_id=acd7b417776a4576bebeb7ae6edcccaa, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.327834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e892028-633f-469c-bcd2-c859359f60ab admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.355528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6de5aaa05dc493792f5fa34f31cd205', 'request_id': 'req-1c351837-cd1e-48d3-8182-81c491d98249'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:25.366935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 913/1827] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:25 2022] PUT /identity/v3/projects/1d538e48e3c4489eaa9ccf5325877992/users/acd7b417776a4576bebeb7ae6edcccaa/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:25.367128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6de5aaa05dc493792f5fa34f31cd205', 'request_id': 'req-1c351837-cd1e-48d3-8182-81c491d98249'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:25.369626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17358-e683-4893-ac2b-e1a456d69b6b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.370023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17358-e683-4893-ac2b-e1a456d69b6b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.370327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17358-e683-4893-ac2b-e1a456d69b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.373643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6de5aaa05dc493792f5fa34f31cd205', 'request_id': 'req-1c351837-cd1e-48d3-8182-81c491d98249'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:25.389475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] The token cache is being invalidate because project e6de5aaa05dc493792f5fa34f31cd205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:25.389788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:25.390430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c351837-cd1e-48d3-8182-81c491d98249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:25.396173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 915/1828] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:25 2022] DELETE /identity/v3/projects/e6de5aaa05dc493792f5fa34f31cd205 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:25.400361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.400798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.401247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.402510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-95d17358-e683-4893-ac2b-e1a456d69b6b None None] MFA Rules not processed for user `acd7b417776a4576bebeb7ae6edcccaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:25.403729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-95d17358-e683-4893-ac2b-e1a456d69b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:25.424487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:25.425691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:25.465680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:25.496254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 916/1829] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:25.500186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 914/1830] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:25.502095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.503842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:25.504769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.505078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.505348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.517566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba23356-1563-44b4-8ca8-2efa870d6d8a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.523461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.524078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.524386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.524699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.525287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.526476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:25.527316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.527670 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:25.538919 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 915/1831] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:25.539984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.550952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f8368-6a47-4c5f-a602-bab6aef9a59a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.556531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.557447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.557832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.558168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.559570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 917/1832] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:25.559689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:25.559689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.560953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.562887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.574868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 916/1833] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:25.579540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.601226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7670c-dc64-46ad-9d15-94d12bb9ac0e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.605603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:25.650464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065edd26-5b0e-4ab0-8ebb-f0bba48ca9cd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.655128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.655755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.656011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.656308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.656934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.658923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.661867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.662483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.662757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.663079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.663718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.665615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.684883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 918/1834] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:25.688868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.696359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 917/1835] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:25.699140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bc351-63f2-4159-a339-3cb79514302c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.700186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.757979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.758591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.758865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.759127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.759762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.761241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604c6-0562-4ec0-b63c-f15f04cde70e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.761417 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.767380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.767599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.767911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.768211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.768848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.770748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.772097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 919/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:25.776258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.786076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44f39-8b8f-4c55-91d5-2acbc962c588 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.791319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.791989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.792298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.792579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] PATH_INFO: `/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff/users/5f804866dac941be9adb843f75b72057/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.793482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 918/1837] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:25.797987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.807639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24fd3b-4d35-464a-903c-de819a90079b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.807784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4e90ec4c98495e9e4396bafe41d3ff, user_id=5f804866dac941be9adb843f75b72057, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.811044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.817173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.817803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.818075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.818345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.818996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.820631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.827339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 920/1838] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:25 2022] PUT /identity/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff/users/5f804866dac941be9adb843f75b72057/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:25.831006 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 919/1839] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:25.831119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.834752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.844866 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02255caf-1280-455a-a9bf-7874016913b5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.851958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:25.894603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd1ff8-7a30-491d-a0e1-66a79e1aa7bc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.899457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.900071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.900405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.900697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.901444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.902907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.903294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.903519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.903798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.904074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.904746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.906324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.913579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 921/1840] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:25.916290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 920/1841] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:25.917718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.920554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:25.927514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f4e61-4d03-4054-8fcd-dabbaa23dbbd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.930047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1429-8496-4570-ab35-098c4f9c8980 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:25.932206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.932917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.933193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.933463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] PATH_INFO: `/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff/users/5f804866dac941be9adb843f75b72057/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.935012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:25.935389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4e90ec4c98495e9e4396bafe41d3ff, user_id=5f804866dac941be9adb843f75b72057, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.935781 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.936159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.936499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] PATH_INFO: `/v3/projects/7a3cec26dfef4943a7e5d440ad150470/users/8f08e8d31eee4279b523c445a867bb14/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.938160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ef1ee-a860-4493-bc58-15045b2f6ba4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.938805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:25.949579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 922/1842] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:25 2022] PUT /identity/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff/users/5f804866dac941be9adb843f75b72057/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:25.953837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3cec26dfef4943a7e5d440ad150470, user_id=8f08e8d31eee4279b523c445a867bb14, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:25.956625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4cd99-a585-436d-833a-33191893495c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:25.960497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.960869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.961105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.977160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 921/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:25 2022] PUT /identity/v3/projects/7a3cec26dfef4943a7e5d440ad150470/users/8f08e8d31eee4279b523c445a867bb14/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:25.987648 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:25.988161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:25.988161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:25.991368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] MFA Rules not processed for user `5f804866dac941be9adb843f75b72057`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:25.992701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.020206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] MFA Rules not processed for user `8f08e8d31eee4279b523c445a867bb14`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:26.021461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.071229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 923/1844] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:26.082947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.084370 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.093151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 922/1845] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:26.097637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.140481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d0866-5f64-45d1-b74c-822e6bbf5695 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.144522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.145132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.145406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.145674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.154934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.156595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.164801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f60e5-3500-4f11-b839-64d94fc8a8ac None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.166177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 924/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:26.169531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.170150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.170427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.170693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.171320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.173339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.202378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 923/1847] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:26 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:26.206335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.217078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7af19-ddf7-4a81-b617-4aef13d7afd9 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.242685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.250617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:26.281759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.282387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.282669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.282939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.283576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.285541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.299616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff8ab-4baa-4107-84ca-794640539c98 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.303746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.304400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.304684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.304957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.308319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 925/1848] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:26 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:26.312153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.314558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:26.322257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b73220-b110-4e47-a3e1-18343ad941bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.327391 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.328055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.328306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.328687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.329334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.330968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.344626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 926/1849] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:26.348700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.358583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0dbf-dd53-4b23-a8fb-388f18dedbb8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.363363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.364049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.364378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.364751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] PATH_INFO: `/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c/users/6aa2a1c3d59b48e795a3abb26036e63e/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.365592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.367084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.377692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 924/1850] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:26.382292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f789e672ea4cdeb6feba0a1cc4f14c, user_id=6aa2a1c3d59b48e795a3abb26036e63e, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.384981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.398438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 927/1851] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:26 2022] PUT /identity/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c/users/6aa2a1c3d59b48e795a3abb26036e63e/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:26.406434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.417801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be275-1a2b-4237-95d1-b48263748f81 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.484445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.485261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.485549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.485819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.486470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.488368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64b578-93bc-47af-ab0b-cd16758ff483 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.502341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 925/1852] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:26.503876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.518822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a8510-5f68-4c03-a835-7d5c950aa0f1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.523575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.524355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.524739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.525007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] PATH_INFO: `/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c/users/6aa2a1c3d59b48e795a3abb26036e63e/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.527021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f789e672ea4cdeb6feba0a1cc4f14c, user_id=6aa2a1c3d59b48e795a3abb26036e63e, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.529661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.539960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 928/1853] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:26 2022] PUT /identity/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c/users/6aa2a1c3d59b48e795a3abb26036e63e/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:26.551074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.551361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.551664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.581514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] MFA Rules not processed for user `6aa2a1c3d59b48e795a3abb26036e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:26.582746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.667205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.671930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 926/1854] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:26.682529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.728748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:26.752230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20338c-a6b6-486e-bd0b-c52e8e7fbea6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.756908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.757555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.757828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.758221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] PATH_INFO: `/v3/users/acd7b417776a4576bebeb7ae6edcccaa` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.758931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf8556-6819-4fc6-b967-97748d407a2e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.759355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd7b417776a4576bebeb7ae6edcccaa)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.761490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.763576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.764646 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.764994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.765431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.777359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:26.779322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dc117-f620-4d23-9ab9-939b4078cbd0 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:26.795169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 927/1855] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:26.825173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd7b417776a4576bebeb7ae6edcccaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:26.833437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd7b417776a4576bebeb7ae6edcccaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:26.850349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd7b417776a4576bebeb7ae6edcccaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:26.852865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd7b417776a4576bebeb7ae6edcccaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:26.854706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:26.861703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd7b417776a4576bebeb7ae6edcccaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:26.869265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 929/1856] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:26 2022] DELETE /identity/v3/users/acd7b417776a4576bebeb7ae6edcccaa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:26.873417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.873655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.873925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.894375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:26.895744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.926748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.928268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.962326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 928/1857] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:26.972104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:26.973433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:26.988864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9fda-059a-4a26-9ad0-bf45a9c57442 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:26.992400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:26.993013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:26.993301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:26.993566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:26.995254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:27.039433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee16468-6aae-4beb-9e1a-43b177f686f3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:27.043226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:27.043956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:27.044259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:27.044527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:27.054321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:27.056451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:27.058775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:27.060230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:27.069204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 929/1858] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:29:27.069731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 930/1859] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:27.338568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:27.340303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:27.362518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748d29-9a56-43ba-99d5-e150ad67d24c service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:27.368030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:27.368949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:27.369222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:27.369488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] PATH_INFO: `/v3/projects/1d538e48e3c4489eaa9ccf5325877992` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:27.370929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d538e48e3c4489eaa9ccf5325877992)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:27.372943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:27.427070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d538e48e3c4489eaa9ccf5325877992', 'request_id': 'req-f69fe1aa-8c65-4c6b-947a-46522521c247'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.433575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d538e48e3c4489eaa9ccf5325877992', 'request_id': 'req-f69fe1aa-8c65-4c6b-947a-46522521c247'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.441834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d538e48e3c4489eaa9ccf5325877992', 'request_id': 'req-f69fe1aa-8c65-4c6b-947a-46522521c247'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.452956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] The token cache is being invalidate because project 1d538e48e3c4489eaa9ccf5325877992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:27.453237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.453841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.455334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 931/1860] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:27 2022] DELETE /identity/v3/projects/1d538e48e3c4489eaa9ccf5325877992 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:27.459361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:27.469168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7dffa-a01a-4276-852f-3ee65d736eb1 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:27.536525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:27.537164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:27.537449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:27.537721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] PATH_INFO: `/v3/users/5f804866dac941be9adb843f75b72057` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:27.538842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f804866dac941be9adb843f75b72057)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:27.540833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:27.562154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:27.591461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f804866dac941be9adb843f75b72057'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.592082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f804866dac941be9adb843f75b72057'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.603851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f804866dac941be9adb843f75b72057'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.612540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f804866dac941be9adb843f75b72057'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.613246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-0c8b82a2-33d1-4318-8fc5-c8ff27ca99a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f804866dac941be9adb843f75b72057'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:27.625642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 930/1861] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:27 2022] DELETE /identity/v3/users/5f804866dac941be9adb843f75b72057 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:27.915747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:27.917490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:27.932295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fe1aa-8c65-4c6b-947a-46522521c247 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:27.964401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:27.998533 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:27.999191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:27.999471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:27.999741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] PATH_INFO: `/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.000890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4e90ec4c98495e9e4396bafe41d3ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.003571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.053905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4e90ec4c98495e9e4396bafe41d3ff', 'request_id': 'req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:28.060631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4e90ec4c98495e9e4396bafe41d3ff', 'request_id': 'req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:28.069935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4e90ec4c98495e9e4396bafe41d3ff', 'request_id': 'req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:28.083298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] The token cache is being invalidate because project bc4e90ec4c98495e9e4396bafe41d3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:28.083600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:28.084348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43b0f26c-c037-4f80-bcef-f894c5ac8fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:28.085912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 932/1862] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:27 2022] DELETE /identity/v3/projects/bc4e90ec4c98495e9e4396bafe41d3ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:28.090902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.091144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.091420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.112701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:28.116901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:28.177573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 931/1863] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:28.185448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.186401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.186401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.207642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:28.208910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:28.217670 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 933/1864] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:28.222034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.232248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c05b4-add2-4a40-9a17-2b673efdc6d4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.236922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.237533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.237810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.238082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.238770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:28.239071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.240710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.252382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 932/1865] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:28.257105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.271763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cfcef-abe3-4368-881e-7e4807a6951a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.278752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.279344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.279615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.279874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.280527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.282409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.313408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 934/1866] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:28.318101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.332713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4669-e224-449e-aec3-d01526e14c41 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.391836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.392381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.392657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.392896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.393535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.395428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.418240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 933/1867] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:28.422415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.433996 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8a781-a120-4126-b91d-b306dd74d039 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.439150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.439715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.439993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.440295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.440945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.442747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.456006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 935/1868] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:28.457434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.468519 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734d84e-41a2-4f34-94d6-c0077edf2106 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.473553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.474214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.474493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.474764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.475467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.477114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aefc55-1cf9-42e2-b0ec-a5ef4cb8cfdc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.488170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 934/1869] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:28.492393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.502478 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b190-f911-40da-9208-02c9a6329641 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.507805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.508488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.508759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.509028 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] PATH_INFO: `/v3/projects/81616879a7814efe98f9f6b6805acc66/users/7b93711bcc704624ac849856659f571a/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.526031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81616879a7814efe98f9f6b6805acc66, user_id=7b93711bcc704624ac849856659f571a, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.528851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.544268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 936/1870] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:28 2022] PUT /identity/v3/projects/81616879a7814efe98f9f6b6805acc66/users/7b93711bcc704624ac849856659f571a/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:28.555463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.555928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.556065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.564480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:28.585974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] MFA Rules not processed for user `7b93711bcc704624ac849856659f571a`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:28.587185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:28.653808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 935/1871] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:28.659351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.721518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:28.726702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ed95-6f2e-49d7-b656-a18993c56e73 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.733828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.734426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.734721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.735009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.735607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.737951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.765769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 937/1872] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:28.770282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.781327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f0b5-8541-4c1a-982d-0fe5a8b8d920 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.843409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.844020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.844348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.844610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.845281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.847183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.871304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 936/1873] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:28 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:28.875395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.885977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d774-5b62-4a06-b022-b088bcc45dc4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.891654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.892281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.892560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.892823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.893463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.895103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.906654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 938/1874] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:28.910666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.921154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ab68-20ac-4d8b-b1a1-cdd4cccdb621 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.927289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.927907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.928240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.928524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.929264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.930878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc063f0-e354-4591-ae85-79d20fc0c0c2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.941618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 937/1875] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:28.946013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:28.956372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a9daa-210f-4d34-b176-5b90d091ff4b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:28.962342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:28.963023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:28.963295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:28.963558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] PATH_INFO: `/v3/projects/1423e6ba1eac4b52a8bed8ce14270ef8/users/d27fc7cd99764a82814c54b00d79b4bc/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:28.978864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1423e6ba1eac4b52a8bed8ce14270ef8, user_id=d27fc7cd99764a82814c54b00d79b4bc, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:28.981543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:28.996390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 939/1876] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:28 2022] PUT /identity/v3/projects/1423e6ba1eac4b52a8bed8ce14270ef8/users/d27fc7cd99764a82814c54b00d79b4bc/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:29.006749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.007047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.007316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.041361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] MFA Rules not processed for user `d27fc7cd99764a82814c54b00d79b4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:29.043208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:29.115198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 938/1877] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:29.120151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.187515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7c68e-b0e6-46da-a0da-fc1f1a39795b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.198854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.199787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.199898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.200185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.200901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.202952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.232697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 940/1878] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:29 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:29.237036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.242898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:29.249816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8690-937e-4b1d-b4aa-3b172f464743 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.310785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.311410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.311696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.311956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.312746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.314649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.342369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 939/1879] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:29 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:29:29.346643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.358021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88e44b-3fc6-436c-a259-eed9a689ee19 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.362781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.363395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.363656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.363923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.364665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.366331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.377872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 941/1880] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:29:29.382167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.392515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cfa57-ceca-4222-b175-1bd39b3ea3d1 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.397152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.397829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.398111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.398385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] PATH_INFO: `/v3/projects/27437242d83e4cfcae8ff6bfdd74c419/users/1554f3dccd0f4e7b9eef752366d3e6fc/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.413958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27437242d83e4cfcae8ff6bfdd74c419, user_id=1554f3dccd0f4e7b9eef752366d3e6fc, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.416932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.431612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 940/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:29 2022] PUT /identity/v3/projects/27437242d83e4cfcae8ff6bfdd74c419/users/1554f3dccd0f4e7b9eef752366d3e6fc/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:29.437984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.452241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f33-fc61-439f-a13f-d346ed685bd8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.513095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.513750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.514019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.514396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.515030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.516975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583b81-eda6-4a5a-b961-f0d429b350df admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.530572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 942/1882] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:29.531770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.544081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2bf29-62f3-47f2-a4e0-cb1337b63557 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.548300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.549139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.549437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.549675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] PATH_INFO: `/v3/projects/27437242d83e4cfcae8ff6bfdd74c419/users/1554f3dccd0f4e7b9eef752366d3e6fc/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.551908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27437242d83e4cfcae8ff6bfdd74c419, user_id=1554f3dccd0f4e7b9eef752366d3e6fc, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.554631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.565766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 941/1883] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:29 2022] PUT /identity/v3/projects/27437242d83e4cfcae8ff6bfdd74c419/users/1554f3dccd0f4e7b9eef752366d3e6fc/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:29.576965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.577228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.577493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.606111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] MFA Rules not processed for user `1554f3dccd0f4e7b9eef752366d3e6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:29.607327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:29.611278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:29.680207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 943/1884] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:29.693168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:29.694996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:29.801195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c4f7-21c7-48c9-8f0a-ac23d4e4f01a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:29.805483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:29.806325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:29.806857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:29.807311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:29.871722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:29.873446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533519-f5aa-4a1c-ad81-6a2545445f3c service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:29.882120 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 942/1885] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:30.813536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:30.815091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:30.883803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834723da-d193-450b-85fe-1e0b9a4c6ca0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:30.889127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:30.889802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:30.890077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:30.890343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] PATH_INFO: `/v3/users/8f08e8d31eee4279b523c445a867bb14` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:30.891897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f08e8d31eee4279b523c445a867bb14)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:30.893884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:30.943024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f08e8d31eee4279b523c445a867bb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:30.950128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f08e8d31eee4279b523c445a867bb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:30.962819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f08e8d31eee4279b523c445a867bb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:30.963572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f08e8d31eee4279b523c445a867bb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:30.971943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f08e8d31eee4279b523c445a867bb14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:30.978292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 944/1886] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:30 2022] DELETE /identity/v3/users/8f08e8d31eee4279b523c445a867bb14 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:30.982952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:30.983417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:30.983508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:31.002719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:31.003920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.008205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:31.070272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 943/1887] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:31.083499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:31.085141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.109639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:31.145960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebdad-e745-4a76-b631-d85e1be3c452 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:31.149818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:31.150435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:31.150713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:31.150983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:31.160427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:31.161857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:31.171775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 945/1888] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:31.442707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:31.444435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.463273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:31.465005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.480989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca5ddc-0deb-40db-8831-e550a3b8b733 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:31.486425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:31.487257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:31.487590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:31.487861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] PATH_INFO: `/v3/projects/7a3cec26dfef4943a7e5d440ad150470` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:31.489091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3cec26dfef4943a7e5d440ad150470)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:31.492960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:31.516019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff185ee-af9e-4f94-a107-e265d3477b0f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:31.521322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:31.521945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:31.522592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:31.523210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:31.526215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.549703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3cec26dfef4943a7e5d440ad150470', 'request_id': 'req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.556977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3cec26dfef4943a7e5d440ad150470', 'request_id': 'req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.567905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3cec26dfef4943a7e5d440ad150470', 'request_id': 'req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.579944 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] The token cache is being invalidate because project 7a3cec26dfef4943a7e5d440ad150470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:31.580594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.581615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.587021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 946/1889] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:31 2022] DELETE /identity/v3/projects/7a3cec26dfef4943a7e5d440ad150470 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:31.588463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:31.590627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:31.607931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:31.610180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:31.631638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 944/1890] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:31.740189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc22bd5-7ffd-4852-b7d0-d353bce0b200 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:31.747618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:31.748327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:31.748610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:31.748885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] PATH_INFO: `/v3/users/6aa2a1c3d59b48e795a3abb26036e63e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:31.750044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa2a1c3d59b48e795a3abb26036e63e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:31.753080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:31.860883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa2a1c3d59b48e795a3abb26036e63e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.862866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:31.888765 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa2a1c3d59b48e795a3abb26036e63e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.925348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa2a1c3d59b48e795a3abb26036e63e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.926165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa2a1c3d59b48e795a3abb26036e63e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.948366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa2a1c3d59b48e795a3abb26036e63e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:31.959865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 947/1891] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:31 2022] DELETE /identity/v3/users/6aa2a1c3d59b48e795a3abb26036e63e => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:32.342974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.372364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849c4d-ab89-4919-bff3-05b247326595 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.431555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.432276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.432563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.432900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] PATH_INFO: `/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.434177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f789e672ea4cdeb6feba0a1cc4f14c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.436306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.443269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:32.458176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.460172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:32.486824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f789e672ea4cdeb6feba0a1cc4f14c', 'request_id': 'req-2dd6c2f6-eb02-4844-84cc-6ace24c79478'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:32.493753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f789e672ea4cdeb6feba0a1cc4f14c', 'request_id': 'req-2dd6c2f6-eb02-4844-84cc-6ace24c79478'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:32.502444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f789e672ea4cdeb6feba0a1cc4f14c', 'request_id': 'req-2dd6c2f6-eb02-4844-84cc-6ace24c79478'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:32.513482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] The token cache is being invalidate because project 09f789e672ea4cdeb6feba0a1cc4f14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:32.513764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:32.514395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2dd6c2f6-eb02-4844-84cc-6ace24c79478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:32.515918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 945/1892] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:32 2022] DELETE /identity/v3/projects/09f789e672ea4cdeb6feba0a1cc4f14c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:32.521734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3d-6d28-463c-8a63-2713d60104d2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.522012 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3d-6d28-463c-8a63-2713d60104d2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.522287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3d-6d28-463c-8a63-2713d60104d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.523330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfac1b-6c37-4419-aed5-96858fb4d5e3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.541749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-0bbdcf3d-6d28-463c-8a63-2713d60104d2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:32.542946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-0bbdcf3d-6d28-463c-8a63-2713d60104d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:32.572149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.572789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.573103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.573341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.575248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:32.606612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 946/1893] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:32.614304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.614588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.614859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.634153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:32.635385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:32.642102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.643561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.644107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 947/1894] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:32.648404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.653458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 948/1895] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:32.657742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f63b-c104-48ac-9650-ad017f4923f0 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.662502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.664467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.664467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.664467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.664467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:32.664771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.666155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.677853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 948/1896] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:29:32.682583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.693164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186d6fd-eddc-4c04-b59d-6d756f33437e service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.697697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.698292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.698576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.698829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.699466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.701400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.732162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 949/1897] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:32 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:32.734753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.745107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6736c2f-320a-4c5a-8eb8-47ee2785c473 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.802675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.803287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.803595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.803863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.804533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.806456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.831770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 949/1898] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:32 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:32.833191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.843949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c9fa-704d-44eb-8e70-0604d53f7d5c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.852255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.852882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.853157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.853427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.854109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.855718 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.858371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:32.869949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 950/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:32.873835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.880220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:32.883911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d37b0-a7f4-459f-b447-085337c43cd5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.889736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.890353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.890822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.891246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.892313 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.894124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df64c8-c939-4da2-8eda-13b1cc0aad32 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.906029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 950/1900] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:32.910188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:32.920347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ca8b-59b9-4235-815b-2f56a4396f61 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:32.925080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:32.925726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.925994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.926257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] PATH_INFO: `/v3/projects/f54b2d15282445d0b9fa45bb74aa21b4/users/d9f14cd4a8d941fbbedd5c5e31e12491/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:32.940939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54b2d15282445d0b9fa45bb74aa21b4, user_id=d9f14cd4a8d941fbbedd5c5e31e12491, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:32.943598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:32.967332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 951/1901] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:32 2022] PUT /identity/v3/projects/f54b2d15282445d0b9fa45bb74aa21b4/users/d9f14cd4a8d941fbbedd5c5e31e12491/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:32.968774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:32.969413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:32.969413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:33.003873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] MFA Rules not processed for user `d9f14cd4a8d941fbbedd5c5e31e12491`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:33.005359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:33.072475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 951/1902] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:33.086407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:33.087873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:33.147355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ed76-c7aa-4f39-89e7-0ee064328ffc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:33.151866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:33.152552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:33.152845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:33.153542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:33.162364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:33.163819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7e164-1b2e-4796-bde3-dc6e35fd1ec1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:33.175412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 952/1903] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:43.068760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=35 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:43.168860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:45.353672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:45.355141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:45.437762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f297c5-a850-4590-852f-957ea7e378f4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:45.442823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:45.443517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:45.443805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:45.444072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] PATH_INFO: `/v3/users/7b93711bcc704624ac849856659f571a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:45.445321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b93711bcc704624ac849856659f571a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:45.447296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:45.502663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b93711bcc704624ac849856659f571a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:45.503508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b93711bcc704624ac849856659f571a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:45.523617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b93711bcc704624ac849856659f571a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:45.535650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b93711bcc704624ac849856659f571a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:45.536381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b93711bcc704624ac849856659f571a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:45.556189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 952/1904] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:45 2022] DELETE /identity/v3/users/7b93711bcc704624ac849856659f571a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:45.560267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:45.560788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:45.561036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:45.584259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:45.589841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:45.671034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 953/1905] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:45.693078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:45.695629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:45.779131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf211bb-ec9d-4485-8a89-c4abe585dc1a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:45.783556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:45.784558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:45.785142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:45.785617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:45.798213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:45.799861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:45.815981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 953/1906] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:46.323057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:46.324639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:46.337599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2996d-04f6-43cb-98a9-546a8f5c6a8c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:46.342752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:46.343929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:46.344248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:46.344521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] PATH_INFO: `/v3/projects/81616879a7814efe98f9f6b6805acc66` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:46.345970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=81616879a7814efe98f9f6b6805acc66)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:46.347878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:46.423645 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81616879a7814efe98f9f6b6805acc66', 'request_id': 'req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.430096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81616879a7814efe98f9f6b6805acc66', 'request_id': 'req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.438673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81616879a7814efe98f9f6b6805acc66', 'request_id': 'req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.449452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] The token cache is being invalidate because project 81616879a7814efe98f9f6b6805acc66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:46.449748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.450361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.452049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 954/1907] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:46 2022] DELETE /identity/v3/projects/81616879a7814efe98f9f6b6805acc66 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:46.456080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:46.466042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552700d-892f-49ed-9049-b6058f46bbf2 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:46.485240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:46.533748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:46.534390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:46.534665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:46.534935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] PATH_INFO: `/v3/users/d27fc7cd99764a82814c54b00d79b4bc` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:46.536083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27fc7cd99764a82814c54b00d79b4bc)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:46.538026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:46.590584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27fc7cd99764a82814c54b00d79b4bc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.591258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27fc7cd99764a82814c54b00d79b4bc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.604659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27fc7cd99764a82814c54b00d79b4bc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.613175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27fc7cd99764a82814c54b00d79b4bc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.613779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27fc7cd99764a82814c54b00d79b4bc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:46.626431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 954/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:46 2022] DELETE /identity/v3/users/d27fc7cd99764a82814c54b00d79b4bc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:46.911948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:46.914153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:46.929274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394fc46-15b3-4f17-b6e4-7c0ef1537fef admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:46.993263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:46.993902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:46.994177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:46.994446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] PATH_INFO: `/v3/projects/1423e6ba1eac4b52a8bed8ce14270ef8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:46.995620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1423e6ba1eac4b52a8bed8ce14270ef8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:46.997542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:47.033902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:47.051647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1423e6ba1eac4b52a8bed8ce14270ef8', 'request_id': 'req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.058298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1423e6ba1eac4b52a8bed8ce14270ef8', 'request_id': 'req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.066863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1423e6ba1eac4b52a8bed8ce14270ef8', 'request_id': 'req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.080124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] The token cache is being invalidate because project 1423e6ba1eac4b52a8bed8ce14270ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:47.080446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.081100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.082660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 955/1909] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:46 2022] DELETE /identity/v3/projects/1423e6ba1eac4b52a8bed8ce14270ef8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:47.086887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:47.088325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:47.101725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629164df-0c04-453a-aeca-0ce29e512256 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:47.167438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:47.168565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.168802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.169065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] PATH_INFO: `/v3/users/1554f3dccd0f4e7b9eef752366d3e6fc` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.170341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1554f3dccd0f4e7b9eef752366d3e6fc)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:47.172339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:47.241281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:47.246461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1554f3dccd0f4e7b9eef752366d3e6fc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.247441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1554f3dccd0f4e7b9eef752366d3e6fc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.273309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1554f3dccd0f4e7b9eef752366d3e6fc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.294284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1554f3dccd0f4e7b9eef752366d3e6fc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.294986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a2f90dc9-11cc-4a4d-aacd-de21b8e20a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1554f3dccd0f4e7b9eef752366d3e6fc'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.317211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 955/1910] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:47 2022] DELETE /identity/v3/users/1554f3dccd0f4e7b9eef752366d3e6fc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:29:47.608559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:47.610166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:47.632004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f99ff6-bb7c-4096-8774-f0fca2deddf5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:47.688797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:47.689468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.689741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.689981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] PATH_INFO: `/v3/projects/27437242d83e4cfcae8ff6bfdd74c419` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.691152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=27437242d83e4cfcae8ff6bfdd74c419)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:47.693094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:47.750327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27437242d83e4cfcae8ff6bfdd74c419', 'request_id': 'req-006f0faf-7141-4d63-bfdd-aae3321f02bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.756745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27437242d83e4cfcae8ff6bfdd74c419', 'request_id': 'req-006f0faf-7141-4d63-bfdd-aae3321f02bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.764793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27437242d83e4cfcae8ff6bfdd74c419', 'request_id': 'req-006f0faf-7141-4d63-bfdd-aae3321f02bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.776042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] The token cache is being invalidate because project 27437242d83e4cfcae8ff6bfdd74c419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:47.776381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.776990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-006f0faf-7141-4d63-bfdd-aae3321f02bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:47.778503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 956/1911] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:47 2022] DELETE /identity/v3/projects/27437242d83e4cfcae8ff6bfdd74c419 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:47.783553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.784044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.784261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.804636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:47.805887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:47.875952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 956/1912] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:47.883855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.884420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.884665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.906246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:47.907460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:47.916625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 957/1913] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:47.920884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:47.930935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104c7e-9734-4523-8302-4992dce443e8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:47.935834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:47.936504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.936787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.937049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.937724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:29:47.938023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:47.939613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:47.952695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 957/1914] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:29:47.957783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:47.968104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6122fe-b5c1-4e62-930b-84e6368fcd76 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:47.976587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:47.977312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:47.977918 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:47.978219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:47.978870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:47.981017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.012774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 958/1915] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:29:47 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:48.013877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:48.029247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eabdca-bd18-40ac-8a68-c4a05e71ce46 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:48.086677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:48.087282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.087561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.087829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.088569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:48.090466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.116377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 958/1916] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:29:48 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:29:48.120764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:48.131152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0d1b8-3c71-4a59-bf0b-36f0a38949d4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:48.135802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:48.136464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.136741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.137009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.137660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:48.139262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.149519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 959/1917] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:48.153390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:48.163593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20237f19-ca73-4f1e-bef3-61cb6c83aaa7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:48.168834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:48.169432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.169705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.169975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.170733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:48.172337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd12c1-2763-49f5-a094-435d3880e1e5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.183216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 959/1918] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:29:48.187418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:48.195612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:48.199683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a58db4-a182-4529-9c68-815fd580c4b5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:48.205120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:48.206087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.206482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.206754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] PATH_INFO: `/v3/projects/9fa6b979d7fa4a9da5f6c9eda82e2789/users/f05572ca6a0c43698a612dbfeab20153/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.224569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa6b979d7fa4a9da5f6c9eda82e2789, user_id=f05572ca6a0c43698a612dbfeab20153, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:48.227236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.242330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 960/1919] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:29:48 2022] PUT /identity/v3/projects/9fa6b979d7fa4a9da5f6c9eda82e2789/users/f05572ca6a0c43698a612dbfeab20153/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:48.252913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2f8e1-b604-4f56-8cc5-1be68a7f5671 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.253192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2f8e1-b604-4f56-8cc5-1be68a7f5671 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.253461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2f8e1-b604-4f56-8cc5-1be68a7f5671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.281848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bfc2f8e1-b604-4f56-8cc5-1be68a7f5671 None None] MFA Rules not processed for user `f05572ca6a0c43698a612dbfeab20153`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:48.283055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bfc2f8e1-b604-4f56-8cc5-1be68a7f5671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:48.299696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:48.365941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 960/1920] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:48.377034 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:48.378777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:48.435228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ce772-f917-4e1a-a28c-543d6a57004a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:48.438763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:48.439372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:48.439641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:48.439901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:48.448075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:48.449466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ac988-3724-450f-a91e-6e82c12529df service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:48.457992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 961/1921] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:50.617679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:50.618155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:50.618311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:50.638981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] MFA Rules not processed for user `b1146abe4f9042968fd32a639cd504bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:50.640248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:50.696526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 961/1922] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:53.024153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a235-37f5-40c2-92d1-49a3fa35e5db None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:53.024652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a235-37f5-40c2-92d1-49a3fa35e5db None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:53.024912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56a235-37f5-40c2-92d1-49a3fa35e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:53.049469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c56a235-37f5-40c2-92d1-49a3fa35e5db None None] MFA Rules not processed for user `b50f86c6209f4a728cf3681a7d0a6765`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:53.050854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c56a235-37f5-40c2-92d1-49a3fa35e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:53.113530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 962/1923] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:54.538029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:54.539621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:54.619492 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046ac56-d26d-4207-84e6-5d4a2660f644 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:54.624208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:54.624855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:54.625130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:54.625397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] PATH_INFO: `/v3/users/d9f14cd4a8d941fbbedd5c5e31e12491` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:54.626532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f14cd4a8d941fbbedd5c5e31e12491)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:54.628535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:54.677525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f14cd4a8d941fbbedd5c5e31e12491'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:54.678178 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f14cd4a8d941fbbedd5c5e31e12491'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:54.688745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:54.689926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f14cd4a8d941fbbedd5c5e31e12491'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:54.698743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f14cd4a8d941fbbedd5c5e31e12491'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:54.699336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f14cd4a8d941fbbedd5c5e31e12491'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:54.711572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 962/1924] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:54 2022] DELETE /identity/v3/users/d9f14cd4a8d941fbbedd5c5e31e12491 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:54.716145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:54.716622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:54.716752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:54.736332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:54.737538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:54.800078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 963/1925] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:29:54.809988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:54.811399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:54.875873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bf276-cdf6-4827-9ef3-cf9f318cab7d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:54.879727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:54.880435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:54.880907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:54.881183 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:54.891808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:54.893245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:54.904513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 963/1926] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:55.225164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:55.226878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:55.245375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2b681-a78d-4de9-8b5f-4b028cd55fca None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:55.251476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:55.252324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:55.252668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:55.253185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] PATH_INFO: `/v3/projects/f54b2d15282445d0b9fa45bb74aa21b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:55.254568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54b2d15282445d0b9fa45bb74aa21b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:55.257171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:55.283263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:55.307422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54b2d15282445d0b9fa45bb74aa21b4', 'request_id': 'req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:55.314129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54b2d15282445d0b9fa45bb74aa21b4', 'request_id': 'req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:55.322900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54b2d15282445d0b9fa45bb74aa21b4', 'request_id': 'req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:55.334189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] The token cache is being invalidate because project f54b2d15282445d0b9fa45bb74aa21b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:55.334472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:55.335107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a74bfb4-55fc-4cec-a7e2-282a7c01790b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:55.336692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 964/1927] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:55 2022] DELETE /identity/v3/projects/f54b2d15282445d0b9fa45bb74aa21b4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:56.376351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:56.378081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:56.393192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e0-91b3-4bec-8666-ded72ba249a6 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:56.455383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:56.456029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:56.456362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:56.456596 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] PATH_INFO: `/v3/users/96dd9688c16a41948e69c0a5fbfa3d88` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:56.457778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dd9688c16a41948e69c0a5fbfa3d88)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:56.459686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:56.508283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dd9688c16a41948e69c0a5fbfa3d88'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:56.508990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dd9688c16a41948e69c0a5fbfa3d88'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:56.521263 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dd9688c16a41948e69c0a5fbfa3d88'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:56.530126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dd9688c16a41948e69c0a5fbfa3d88'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:56.530759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dd9688c16a41948e69c0a5fbfa3d88'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:56.543536 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 964/1928] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:29:56 2022] DELETE /identity/v3/users/96dd9688c16a41948e69c0a5fbfa3d88 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:29:56.548537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:56.549035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:56.549035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:56.568798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:29:56.569998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:56.632517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 965/1929] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:29:56.643260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:56.644840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:56.650666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:29:56.704659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330769f-56a8-4262-869b-50a6b9c8b29f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:56.708438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:56.709035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:56.709307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:56.709570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:56.719476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:56.720920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed838326-6761-4216-a6d9-2bb749765900 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:56.730509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 965/1930] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:29:57.001649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:29:57.003440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:29:57.017687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a945f6-0e48-4ff1-8e94-65e91c6d2a7a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:29:57.023786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:29:57.024477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:29:57.024747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:29:57.025099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] PATH_INFO: `/v3/projects/5d8e73317ddd440a97287c6c745bb387` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:29:57.026217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8e73317ddd440a97287c6c745bb387)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:29:57.028214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:29:57.080955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8e73317ddd440a97287c6c745bb387', 'request_id': 'req-1e222120-dc06-4be5-81c0-48ce226d7b53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:57.087523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8e73317ddd440a97287c6c745bb387', 'request_id': 'req-1e222120-dc06-4be5-81c0-48ce226d7b53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:57.098229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8e73317ddd440a97287c6c745bb387', 'request_id': 'req-1e222120-dc06-4be5-81c0-48ce226d7b53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:57.108600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] The token cache is being invalidate because project 5d8e73317ddd440a97287c6c745bb387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:29:57.108881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:57.109503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e222120-dc06-4be5-81c0-48ce226d7b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:29:57.110998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 966/1931] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:29:57 2022] DELETE /identity/v3/projects/5d8e73317ddd440a97287c6c745bb387 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:06.726386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=25 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:07.105684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=69,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:12.854829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b547096-4831-4a4d-9260-29a19254fe06 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:12.855241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b547096-4831-4a4d-9260-29a19254fe06 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:12.855333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b547096-4831-4a4d-9260-29a19254fe06 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:12.856676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 966/1932] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 11 13:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:30:12.862228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a6e7d-2bfe-4e7e-b3c2-8ab0ac63da72 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:12.862678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a6e7d-2bfe-4e7e-b3c2-8ab0ac63da72 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:12.862951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a6e7d-2bfe-4e7e-b3c2-8ab0ac63da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:12.890541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f1a6e7d-2bfe-4e7e-b3c2-8ab0ac63da72 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:12.892139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f1a6e7d-2bfe-4e7e-b3c2-8ab0ac63da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:12.966206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 967/1933] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 11 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:12.974075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b547096-4831-4a4d-9260-29a19254fe06 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:12.988547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b547096-4831-4a4d-9260-29a19254fe06 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:12.994112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:12.996080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:12.996080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:12.996080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:12.997470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:13.011768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:13.013412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:13.025407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 967/1934] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:30:22.957812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=29 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:23.020793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:23.164688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:23.165000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:23.165300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:23.186506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:23.187699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:23.242434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 968/1935] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:23.285378 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:23.286997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:23.368248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd07cd6-3343-4274-93a4-d23bdc4daa92 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:23.376754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:23.377363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:23.377644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:23.377907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:23.386961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:23.388577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7cd10-d9f4-4655-bb8b-382097e56311 service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:23.397347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 968/1936] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:30:33.235124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=25 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:33.394027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=32 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:40.152166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:40.153967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.222662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103a800-27b7-45aa-9846-7f5c621ee187 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:40.227560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:40.228238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.228519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.228783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] PATH_INFO: `/v3/users/f05572ca6a0c43698a612dbfeab20153` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.229990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05572ca6a0c43698a612dbfeab20153)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:40.231922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:40.283280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05572ca6a0c43698a612dbfeab20153'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.290270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05572ca6a0c43698a612dbfeab20153'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.302594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05572ca6a0c43698a612dbfeab20153'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.303320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05572ca6a0c43698a612dbfeab20153'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.312697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-698efaef-60a3-4fa0-a9c5-cfe559cd0b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05572ca6a0c43698a612dbfeab20153'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.318672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 969/1937] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:30:40 2022] DELETE /identity/v3/users/f05572ca6a0c43698a612dbfeab20153 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:40.323307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.323808 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.323927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.343604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:40.344878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.361497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.361786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.362063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.384137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:40.385522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.404495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 969/1938] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:40.415282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:40.416657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.441857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 970/1939] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:30:40.455083 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:40.456449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.475634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe770d3-61bb-4ddc-9925-2e7121d83170 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:40.479344 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:40.479967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.480278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.480553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.483715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:40.490318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:40.491787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:40.501669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 970/1940] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:30:40.527896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c76803-a80b-4e44-8468-19bb1171ac7e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:40.533047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:40.533665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.533938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.534205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.542717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:40.544350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9259ee-1718-4d87-9d49-7701485dbefd service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:40.552537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 971/1941] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:30:40.805599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:40.807349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:40.823337 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0dff-168d-4742-8ef9-d098db61e15b service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:40.833382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:40.834089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.834370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.834710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] PATH_INFO: `/v3/projects/9fa6b979d7fa4a9da5f6c9eda82e2789` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.836090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa6b979d7fa4a9da5f6c9eda82e2789)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:40.838061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:40.889572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa6b979d7fa4a9da5f6c9eda82e2789', 'request_id': 'req-660b4100-814f-4d21-a4a4-eb21bf2c35c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.896719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa6b979d7fa4a9da5f6c9eda82e2789', 'request_id': 'req-660b4100-814f-4d21-a4a4-eb21bf2c35c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.905776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa6b979d7fa4a9da5f6c9eda82e2789', 'request_id': 'req-660b4100-814f-4d21-a4a4-eb21bf2c35c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.917839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] The token cache is being invalidate because project 9fa6b979d7fa4a9da5f6c9eda82e2789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:30:40.918126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.918760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-660b4100-814f-4d21-a4a4-eb21bf2c35c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:40.920294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 971/1942] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:30:40 2022] DELETE /identity/v3/projects/9fa6b979d7fa4a9da5f6c9eda82e2789 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:40.925517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:40.925805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:40.926100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:40.949134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:40.950445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:41.021222 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 972/1943] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:41.029240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.029526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.029930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.050223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:41.051422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:41.060599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 972/1944] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:30:41.065079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.075418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a758a8-f66e-4388-9830-7942611b18be None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.081015 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.081610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.081910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.082178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.082836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:30:41.083164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.084832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.097131 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 973/1945] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:30:41.101385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.111014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e1e6-65a3-4bb8-a3c0-ab78fbe91201 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.115814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.116484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.116755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.117022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.117723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.119639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.141138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:41.150500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 973/1946] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:30:41 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:30:41.154561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.164946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a20a-1954-432a-901a-79bb0011695c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.223180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.223792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.224067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.224399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.225037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.226973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.251055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 974/1947] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:30:41 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:30:41.255226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.266801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4f66a-6e8b-4327-86c8-a86b735b320c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.271569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.272227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.272505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.272774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.273437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.275023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.284960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 974/1948] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:41.288957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.299154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ebde3-00de-4c52-bebe-5dd1e9a18f90 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.304107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.304743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.305011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.305283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.305940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.307558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0d22f-5f17-4db7-861b-58b86725d062 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.318517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 975/1949] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:41.323803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.334153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd7dc-bcc2-490c-bcdc-d3e34c6915a6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.339527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.340205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.340498 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.340763 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] PATH_INFO: `/v3/projects/07b1cc93b68340b787ee0c99f1bf7c68/users/ee084e4a27a94d409653f36e786af9e7/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.355388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b1cc93b68340b787ee0c99f1bf7c68, user_id=ee084e4a27a94d409653f36e786af9e7, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.358045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.372335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 975/1950] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:30:41 2022] PUT /identity/v3/projects/07b1cc93b68340b787ee0c99f1bf7c68/users/ee084e4a27a94d409653f36e786af9e7/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:41.377454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ec3b-f62a-4d6e-b74c-99ca40a18e16 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.377753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ec3b-f62a-4d6e-b74c-99ca40a18e16 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.377998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ec3b-f62a-4d6e-b74c-99ca40a18e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.399869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0f14ec3b-f62a-4d6e-b74c-99ca40a18e16 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:41.401173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f14ec3b-f62a-4d6e-b74c-99ca40a18e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:41.415312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:41.464081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 976/1951] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:41.473715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:41.475544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:41.533607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263e0c0-8600-4fad-a955-d712f9d7045b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:41.537409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:41.538027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:41.538297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:41.538561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:41.547565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:41.549070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeffc68-aac3-42ed-9d18-1f6a335bef0a service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:41.566111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 976/1952] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:30:42.725745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:42.726267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:42.726267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:42.732095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 977/1953] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:30:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:30:42.738669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:42.740408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:42.740905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:42.778458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] MFA Rules not processed for user `eabb880b598b47c0b7d7f969d4eebffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:42.782043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:42.831105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:42.833479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:42.863166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad8317-6ad2-43fb-aa3f-52bbc9e1c4b5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:42.882135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:42.884292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:42.886685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:42.887162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] PATH_INFO: `/v3/users/b1146abe4f9042968fd32a639cd504bc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:42.891037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1146abe4f9042968fd32a639cd504bc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:42.893643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:42.950229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 977/1954] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:30:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:42.963486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:42.966351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:42.989647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a10f9-62b1-4f11-800e-e59c0ee0cac5 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:43.000585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:43.001244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.001515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.001778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.010577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1146abe4f9042968fd32a639cd504bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.018283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:43.019979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:43.029210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1146abe4f9042968fd32a639cd504bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.035489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 978/1955] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:30:43.049745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1146abe4f9042968fd32a639cd504bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.050894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1146abe4f9042968fd32a639cd504bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.064681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82f18e2-70c5-448a-b426-ca6cd3a1829c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1146abe4f9042968fd32a639cd504bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.078977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 978/1956] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:30:42 2022] DELETE /identity/v3/users/b1146abe4f9042968fd32a639cd504bc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:43.374180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:43.413888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb230e0-640b-4f48-904e-1e7f9606c343 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:43.480035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:43.546678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:43.547370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.547647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.547917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] PATH_INFO: `/v3/projects/05aa2418cebe4625b97d8ef5baf70a35` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.551233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05aa2418cebe4625b97d8ef5baf70a35)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:43.553222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:43.641619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05aa2418cebe4625b97d8ef5baf70a35', 'request_id': 'req-38104bda-8b01-448e-9765-74aba815647d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.655221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05aa2418cebe4625b97d8ef5baf70a35', 'request_id': 'req-38104bda-8b01-448e-9765-74aba815647d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.671636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05aa2418cebe4625b97d8ef5baf70a35', 'request_id': 'req-38104bda-8b01-448e-9765-74aba815647d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.694857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] The token cache is being invalidate because project 05aa2418cebe4625b97d8ef5baf70a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:30:43.695142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.698556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-38104bda-8b01-448e-9765-74aba815647d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:30:43.700478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 979/1957] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:30:43 2022] DELETE /identity/v3/projects/05aa2418cebe4625b97d8ef5baf70a35 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:43.707042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.707042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.707042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.738549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:43.740149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:43.814270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 979/1958] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:43.821905 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.822476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.822476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.854929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:43.856291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:43.866320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 980/1959] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:43.870411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:43.881769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc524-466f-4098-9b19-6b645e2c0b58 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:43.886952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:43.887736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.888125 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.888509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.889298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:30:43.889752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:43.891934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:43.908335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 980/1960] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:30:43.910064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:43.921725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a88e7-9f81-43bc-81dc-3d1e122af4be None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:43.927025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:43.927668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:43.927947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:43.928367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:43.929035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:43.930956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:43.970291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 981/1961] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:30:43 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:30:43.975482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:43.987193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0de72-79ba-40f3-84ec-bdd88ec46cff admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:44.009951 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:44.063037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:44.063893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.064342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.064553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.065183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:44.067205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:44.099933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 981/1962] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:30:43 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:30:44.104699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:44.117767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc7a61-81e7-46f3-8370-7ac065a0e508 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:44.124392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:44.125436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.125835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.126205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.127169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:44.129333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:44.145824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 982/1963] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:44.148823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:44.161898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebda11-693c-48e7-af95-5ebf78805d7f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:44.167409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:44.168484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.168861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.169224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.169988 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:44.171958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e368-73f0-4eaf-bdd4-d23e02159119 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:44.186020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 982/1964] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:44.190583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:44.207824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09209ded-0aeb-4561-93ee-2988d4b393be admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:44.212933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:44.213782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.214060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.214330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] PATH_INFO: `/v3/projects/192799a3ba9a4c0e981f6813a0341768/users/71b6993791904a50a94ba0f980c9c0c6/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.234149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=192799a3ba9a4c0e981f6813a0341768, user_id=71b6993791904a50a94ba0f980c9c0c6, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:44.237722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:44.265232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 983/1965] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:30:44 2022] PUT /identity/v3/projects/192799a3ba9a4c0e981f6813a0341768/users/71b6993791904a50a94ba0f980c9c0c6/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:44.270071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde718f-2f29-4fd2-b22f-9101f56256d0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.270488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde718f-2f29-4fd2-b22f-9101f56256d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.270858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde718f-2f29-4fd2-b22f-9101f56256d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.300488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0cde718f-2f29-4fd2-b22f-9101f56256d0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:44.301854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0cde718f-2f29-4fd2-b22f-9101f56256d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:44.383324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 983/1966] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:44.398804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:44.400635 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:44.487015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056daf2-dfee-4a27-a6cd-7d4e00c28b5c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:44.495365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:44.496239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:44.496622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:44.496993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:44.508205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:44.511077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:44.523235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 984/1967] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:30:45.679838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2839ef-def7-4813-ad59-45c165b2ace6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:45.680385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2839ef-def7-4813-ad59-45c165b2ace6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:45.680759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2839ef-def7-4813-ad59-45c165b2ace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:45.722571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ba2839ef-def7-4813-ad59-45c165b2ace6 None None] MFA Rules not processed for user `71b6993791904a50a94ba0f980c9c0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:45.723824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ba2839ef-def7-4813-ad59-45c165b2ace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:45.848158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 984/1968] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:45.857395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:45.859456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:45.895617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:45.975460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4345191-d236-4a65-9b73-8a4287113d94 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:45.998182 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:45.998493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:45.998907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:45.999353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:46.011520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:46.013426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:46.029190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 985/1969] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:30:46.808932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:46.809421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:46.809502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:46.849073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] MFA Rules not processed for user `ee084e4a27a94d409653f36e786af9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:46.850467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:46.885184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:46.955603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 985/1970] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:46.959917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:46.966656 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:46.980553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35718a1f-74dd-4d33-a39d-506efce3d72e service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:46.987121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:46.987826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:46.988417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:46.988691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:46.989506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:46.991545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.028310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 986/1971] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:30:46 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:30:47.031253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:47.047692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9087a-42ce-47a9-88e4-e528f8c10268 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:47.118931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:47.119657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.120025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.120392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.121190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:47.123141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.161040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 986/1972] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:30:47 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:30:47.165903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:47.179764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f66c0b-e53b-4d97-aad8-e1211e9ee5fb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:47.186683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:47.187402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.187711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.187997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.190901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:47.191682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.205384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 987/1973] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:47.210419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:47.226231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429078-4e90-4872-b4a9-6b28057dfd17 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:47.240141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:47.241079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.241353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.241770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] PATH_INFO: `/v3/projects/be0e7e9dc2e741b3922a0200f9786605/users/e00aa4c96de546ac809c55947f427488/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.259132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0e7e9dc2e741b3922a0200f9786605, user_id=e00aa4c96de546ac809c55947f427488, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:47.262321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.279107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 987/1974] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:30:47 2022] PUT /identity/v3/projects/be0e7e9dc2e741b3922a0200f9786605/users/e00aa4c96de546ac809c55947f427488/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:47.282563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:47.294149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d3fda-9044-41f2-af08-ad224c678614 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:47.393055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:47.393776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.394121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.394450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.395258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:47.397168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33e60b-2d12-45d5-83d0-40e0827472ce admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.412733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 988/1975] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:30:47.417031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:47.429209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d12550-b43f-4ccc-a57c-949c250db51f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:47.438365 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:47.439062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.439340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.439606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] PATH_INFO: `/v3/projects/be0e7e9dc2e741b3922a0200f9786605/users/e00aa4c96de546ac809c55947f427488/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.443703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0e7e9dc2e741b3922a0200f9786605, user_id=e00aa4c96de546ac809c55947f427488, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:47.447234 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf354b-9a06-4a53-9e63-721f41f7910b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:47.460170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 988/1976] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:30:47 2022] PUT /identity/v3/projects/be0e7e9dc2e741b3922a0200f9786605/users/e00aa4c96de546ac809c55947f427488/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:30:47.576810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:47.577046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:47.577267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:47.607851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] MFA Rules not processed for user `71b6993791904a50a94ba0f980c9c0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:47.609171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:47.704173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 989/1977] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:55.090749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955d6a-a849-42d3-af68-bfc6a150ee83 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:55.091235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955d6a-a849-42d3-af68-bfc6a150ee83 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:55.091537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955d6a-a849-42d3-af68-bfc6a150ee83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:55.122056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8e955d6a-a849-42d3-af68-bfc6a150ee83 None None] MFA Rules not processed for user `e00aa4c96de546ac809c55947f427488`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:55.124063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8e955d6a-a849-42d3-af68-bfc6a150ee83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:55.199802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:55.223124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 989/1978] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:30:55.230255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:55.232376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:55.271746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:30:55.302044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765f852-85b5-4992-92aa-ec085b2abb48 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:55.312758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:55.313792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:55.316972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:55.316972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:55.425363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:55.431839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:55.444889 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 990/1979] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:30:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:30:55.652420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653218-dff6-4b86-8ca1-63fa8e4b080c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:55.652420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653218-dff6-4b86-8ca1-63fa8e4b080c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:55.652420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653218-dff6-4b86-8ca1-63fa8e4b080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:55.673448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-16653218-dff6-4b86-8ca1-63fa8e4b080c None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:30:55.674810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-16653218-dff6-4b86-8ca1-63fa8e4b080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:55.744353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 990/1980] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:30:55.754939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:30:55.756391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:30:55.828647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c04e3-2b58-43d4-bd38-ef7cec12142c service glance] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:30:55.833742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:30:55.834389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:30:55.834660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:30:55.834929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:30:55.843840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:30:55.845466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e2e6-6742-444e-9282-21f9e5db22be service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:30:55.854482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 991/1981] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:30:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:31:05.731445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=27 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:31:05.751862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774dca9-ab08-4252-8134-5b7261ab33db None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:05.752243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774dca9-ab08-4252-8134-5b7261ab33db None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:05.752586 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774dca9-ab08-4252-8134-5b7261ab33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:05.773692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6774dca9-ab08-4252-8134-5b7261ab33db None None] MFA Rules not processed for user `ee084e4a27a94d409653f36e786af9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:31:05.774876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6774dca9-ab08-4252-8134-5b7261ab33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:31:05.785801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 991/1982] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:31:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:31:05.852762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=67 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:31:06.649731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:06.650079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:06.650467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:06.676933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] MFA Rules not processed for user `2fc3a4b9afd14409b3aef17f49aafd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:31:06.678399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:31:06.768515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 992/1983] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:31:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:31:08.306997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107224a-a434-4e06-8923-02cb87570a28 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:08.307522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107224a-a434-4e06-8923-02cb87570a28 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:08.307522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107224a-a434-4e06-8923-02cb87570a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:08.327451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0107224a-a434-4e06-8923-02cb87570a28 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:31:08.328698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0107224a-a434-4e06-8923-02cb87570a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:31:08.342650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 992/1984] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:31:08.355411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:31:08.368376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b0c11-0391-4f08-bbd1-b0c8898718fe None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:31:08.373730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:31:08.374408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:08.374686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:08.374951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:08.386150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:31:08.387943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:31:08.407903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 993/1985] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:31:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:31:18.321712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:31:18.395010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=31 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:31:33.174902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:33.175650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:33.175650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:33.214124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:31:33.218957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:31:33.240334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 993/1986] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:31:33.258637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:31:33.270010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142e8f6-82d6-4280-b082-48b258eac87e service swift] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:31:33.275479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:31:33.276110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:31:33.276417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:31:33.276690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:31:33.286884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:31:33.288747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7092-abaf-4d0d-bed2-93a96c567c3d service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:31:33.298977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 994/1987] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:31:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:31:43.200755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:31:43.295377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:32:04.922379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:04.923910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:04.983082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133282b6-6a7a-4832-92ef-89b215166abc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:04.986777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:04.987531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:04.987875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:04.988276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:04.990432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:05.069287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:05.071028 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e1a2f-4d52-4e62-ba5c-72febd110a63 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:05.083802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 994/1988] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:05.793524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:05.794112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:05.794430 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:05.796109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 995/1989] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:32:05.803261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927a9e-8fd9-427b-a224-e04988760dc2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:05.803657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927a9e-8fd9-427b-a224-e04988760dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:05.804031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927a9e-8fd9-427b-a224-e04988760dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:05.826999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-41927a9e-8fd9-427b-a224-e04988760dc2 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:05.829303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-41927a9e-8fd9-427b-a224-e04988760dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:05.839406 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 995/1990] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:32:05.852227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:05.854090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:05.867689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb9bdb-a965-4483-bf8c-56e0f607576f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:05.871791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:05.872754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:05.873256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:05.873715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:05.885742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:05.887835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3ec20-8754-4230-a3b5-4e88290efcd0 service glance] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:05.903065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 996/1991] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:05.995277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d13d-bfcb-4853-a1ff-2e04bb6b76f0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:05.995734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d13d-bfcb-4853-a1ff-2e04bb6b76f0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:05.995863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d13d-bfcb-4853-a1ff-2e04bb6b76f0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:05.997271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 996/1992] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:32:06.006439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e93ee-6419-4cb4-8147-e9cebe46e708 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:06.007014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e93ee-6419-4cb4-8147-e9cebe46e708 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:06.007282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e93ee-6419-4cb4-8147-e9cebe46e708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:06.028595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-715e93ee-6419-4cb4-8147-e9cebe46e708 None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:06.029835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-715e93ee-6419-4cb4-8147-e9cebe46e708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:06.036674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 997/1993] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:32:06.048892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828d13d-bfcb-4853-a1ff-2e04bb6b76f0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:06.061219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828d13d-bfcb-4853-a1ff-2e04bb6b76f0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:06.066442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:06.067026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:06.067303 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:06.067575 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:06.076283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:06.077802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac3b5c-009e-42f7-91e8-d845501f2776 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:06.087278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 997/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:06.176719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f494-81f7-4a0b-8ad3-84aa89c86bcb None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:06.177210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f494-81f7-4a0b-8ad3-84aa89c86bcb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:06.177554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f494-81f7-4a0b-8ad3-84aa89c86bcb None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:06.179014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 998/1995] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:32:06.184545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:06.184893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:06.185167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:06.207675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:06.209342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:06.222507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 998/1996] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:32:06.233813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9f494-81f7-4a0b-8ad3-84aa89c86bcb None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:06.249057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9f494-81f7-4a0b-8ad3-84aa89c86bcb None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:06.254267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:06.255279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:06.255738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:06.256205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:06.269791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:06.271524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:06.292694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 999/1997] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:08.267688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:08.277901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b0af-70c2-4a41-a799-7fd63ae10b25 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:08.282115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:08.282740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:08.283114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:08.283387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:08.285268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:08.298892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:08.300364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:08.311395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 999/1998] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:11.909147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:11.910538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:11.925345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b50c8-d0e1-4137-afd1-89ccdb0eeee8 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:11.931374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:11.931976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:11.932281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:11.932545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:11.934317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:11.999215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:12.001711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5349-c04f-41c0-a39f-f93edde5cbd7 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:12.014074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1000/1999] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:12.751512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:12.771172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c766-d0af-41b5-91dd-67623c4e2465 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:12.778578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:12.779622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:12.779934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:12.780284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:12.793959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:12.796399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:12.812431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1000/2000] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:18.245343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:18.245849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:18.245970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:18.265708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:18.266909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:18.274715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1001/2001] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:32:18.288538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:18.300613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5239-52cf-4a1d-b368-e664f53937cf service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:18.306266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:18.306888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:18.307338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:18.307795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:18.317088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:18.319018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8689460-bb9b-4d09-a0fa-71584c0c688e service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:18.328287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1001/2002] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:32:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:32:23.160775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:23.162143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:23.230170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e1e42-8a9f-4a5b-8739-d611a5ce5fae None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:23.234593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:23.236222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:23.236513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:23.236788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:23.238661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:23.325751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:23.327975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:23.338572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1002/2003] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:28.324873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:32:33.333639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=94 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:32:35.064952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:35.065475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:35.065475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:35.085249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] MFA Rules not processed for user `71b6993791904a50a94ba0f980c9c0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:35.086455 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:35.094354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1002/2004] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:32:35.311019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:35.321624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6945a-4883-4464-8671-ffff682bbb13 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:35.327921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:35.329013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:35.329531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:35.329804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:35.332085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:35.343404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:35.344874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:35.354145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1003/2005] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:35.367090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:35.377047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb2baa-fc0a-47d4-9900-b864631f4ee9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:35.380734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:35.381353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:35.381630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:35.381900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:35.383619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:35.394516 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:35.395958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:35.403939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1003/2006] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:35.726374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:35.728103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:35.804680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0b81c-ef6e-478a-9b1f-e4aea97d458e service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:35.812550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:35.813469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:35.813928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:35.814291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:35.817209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:35.832502 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:35.834073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5834-15eb-442a-9809-99603c44da3e service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:35.844109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1004/2007] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:40.589191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:40.605226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e0c80-e0c9-4c09-8f9c-0f3f22c890b5 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:40.611641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:40.612307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:40.612787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:40.613057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:40.615212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:40.628579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:40.630177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:40.642264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1004/2008] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:32:45.839738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=38 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:32:50.636781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=15 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:32:53.314244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f28a-d5dd-4eea-9970-72d979950cdf None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:53.314738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f28a-d5dd-4eea-9970-72d979950cdf None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:53.314847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f28a-d5dd-4eea-9970-72d979950cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:53.336322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c430f28a-d5dd-4eea-9970-72d979950cdf None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:32:53.337518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c430f28a-d5dd-4eea-9970-72d979950cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:32:53.346109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1005/2009] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:32:53.369173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:32:53.382784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca80c44-cc6e-49d5-977d-0f8d1b80f97a service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:32:53.389396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:32:53.390121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:32:53.390321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:32:53.390594 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:32:53.400551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:32:53.402133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:32:53.410457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1005/2010] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:33:03.329589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:03.407273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:06.001291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:33:06.001749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:33:06.001855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:33:06.022008 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] MFA Rules not processed for user `b50f86c6209f4a728cf3681a7d0a6765`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:33:06.023234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:33:06.031177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1006/2011] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:33:15.778447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:33:15.779984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:33:15.796145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ee0d-685d-429e-aa3d-e3fc09235551 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:33:15.805420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:33:15.806066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:33:15.806517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:33:15.806779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:33:15.808625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:33:15.820982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:33:15.822576 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:33:15.832982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1006/2012] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:33:16.016274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:25.828823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:49.229486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:33:49.292695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1dad0-f325-4ea8-88ea-b5b406fed8f8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:33:49.297873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:33:49.298504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:33:49.298777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:33:49.299043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] PATH_INFO: `/v3/users/71b6993791904a50a94ba0f980c9c0c6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:33:49.300310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b6993791904a50a94ba0f980c9c0c6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:33:49.302233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:33:49.352282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b6993791904a50a94ba0f980c9c0c6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.352943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b6993791904a50a94ba0f980c9c0c6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.365483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b6993791904a50a94ba0f980c9c0c6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.374716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b6993791904a50a94ba0f980c9c0c6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.375338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b6993791904a50a94ba0f980c9c0c6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.388811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1007/2013] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:33:49 2022] DELETE /identity/v3/users/71b6993791904a50a94ba0f980c9c0c6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:33:49.574089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:33:49.585790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edce604-2e97-4cea-989f-562d9ba50f9d service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:33:49.649696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:33:49.650344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:33:49.650616 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:33:49.650883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] PATH_INFO: `/v3/projects/192799a3ba9a4c0e981f6813a0341768` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:33:49.652077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192799a3ba9a4c0e981f6813a0341768)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:33:49.654072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:33:49.702707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192799a3ba9a4c0e981f6813a0341768', 'request_id': 'req-39183734-83a7-4077-aaae-e8411c5a9fc1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.709658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192799a3ba9a4c0e981f6813a0341768', 'request_id': 'req-39183734-83a7-4077-aaae-e8411c5a9fc1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.718830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192799a3ba9a4c0e981f6813a0341768', 'request_id': 'req-39183734-83a7-4077-aaae-e8411c5a9fc1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.729766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] The token cache is being invalidate because project 192799a3ba9a4c0e981f6813a0341768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:33:49.730072 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.730709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:33:49.732326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1007/2014] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:33:49 2022] DELETE /identity/v3/projects/192799a3ba9a4c0e981f6813a0341768 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:33:59.385254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:59.612179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:33:59.613977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:33:59.689915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192961-adb3-400f-92a9-553416b701a8 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:33:59.726681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:33:59.757184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:33:59.757813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:33:59.758095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:33:59.758360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:33:59.760254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:33:59.819554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:33:59.821240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbccf1-58cc-4066-b613-e929e67a1363 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:33:59.830709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1008/2015] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:33:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:09.826111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=84 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:19.848699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:19.850737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:19.868219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183734-83a7-4077-aaae-e8411c5a9fc1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:19.876786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:19.877432 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:19.877705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:19.877966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:19.879896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:19.963426 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:19.965279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:19.980321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1008/2016] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:25.006804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5d4a-c9d2-4cc5-9ce2-6b9ff04c6496 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:25.007244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5d4a-c9d2-4cc5-9ce2-6b9ff04c6496 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:25.007342 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5d4a-c9d2-4cc5-9ce2-6b9ff04c6496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:25.028299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-05dc5d4a-c9d2-4cc5-9ce2-6b9ff04c6496 None None] MFA Rules not processed for user `d439d54a4f2348d78ddb84691f2a4f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:25.029539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-05dc5d4a-c9d2-4cc5-9ce2-6b9ff04c6496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:25.089264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1009/2017] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:25.454808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:25.456262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:25.515227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9d66-ecc6-45f1-84a5-1c31b71755cd service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:25.518976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:25.519793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:25.520191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:25.520579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:25.522453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:25.532329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:25.533766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:25.541950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1009/2018] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:35.081722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=25 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:35.538277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=54 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:35.678794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:35.679459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:35.679739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:35.709326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] MFA Rules not processed for user `2fc3a4b9afd14409b3aef17f49aafd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:35.710666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:35.718815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1010/2019] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:37.275938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:37.292791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecdc95-66e5-446d-aefc-6bb61f35ed3b service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:37.298831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:37.299452 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:37.299737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:37.300008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:37.301860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:37.314371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:37.315964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:37.326986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1010/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:45.702922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:47.322342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:50.629930 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:50.631529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:50.643772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945193a-f315-45dc-958b-7c82d47f34ea None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:50.647335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:50.647955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:50.648297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:50.648575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:50.650461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:50.715058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:50.716598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:50.726570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1011/2021] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:54.234879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:54.236390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:54.253369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1fde3-142f-4287-a174-1076359f6c55 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:54.257869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:54.258518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.258793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.259058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] PATH_INFO: `/v3/users/b50f86c6209f4a728cf3681a7d0a6765` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.260194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50f86c6209f4a728cf3681a7d0a6765)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:54.262569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:54.311840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50f86c6209f4a728cf3681a7d0a6765'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.318886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50f86c6209f4a728cf3681a7d0a6765'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.330851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50f86c6209f4a728cf3681a7d0a6765'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.331552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50f86c6209f4a728cf3681a7d0a6765'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.340339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbbf203d-098c-4a86-80ed-1fb95cd921bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50f86c6209f4a728cf3681a7d0a6765'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.346781 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1011/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:34:54 2022] DELETE /identity/v3/users/b50f86c6209f4a728cf3681a7d0a6765 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:54.533206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:54.543752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bfe3f-0815-4ac9-89d6-a847902c5903 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:54.602394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:54.603035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.603310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.603582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] PATH_INFO: `/v3/projects/46f14bd1e0fd419383ac2bf2ab87e6d0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.604726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f14bd1e0fd419383ac2bf2ab87e6d0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:54.606649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:54.657653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f14bd1e0fd419383ac2bf2ab87e6d0', 'request_id': 'req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.663968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f14bd1e0fd419383ac2bf2ab87e6d0', 'request_id': 'req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.673907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f14bd1e0fd419383ac2bf2ab87e6d0', 'request_id': 'req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.684828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] The token cache is being invalidate because project 46f14bd1e0fd419383ac2bf2ab87e6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:34:54.685104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.685715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3a852a94-1e53-4f9d-abe9-a5e535edf3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:54.687248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1012/2023] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:34:54 2022] DELETE /identity/v3/projects/46f14bd1e0fd419383ac2bf2ab87e6d0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:54.692341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.692625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.692894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.712971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:54.714170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:54.775073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1012/2024] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:34:54.783918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.784240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.784514 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.804609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:54.805818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:54.814157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1013/2025] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:34:54.818466 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:54.828626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91add-3e59-4016-8083-03b4eff6acea None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:54.834499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:54.835097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.835368 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.835629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.836302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:34:54.836567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:54.838195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:54.850287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1013/2026] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:34:54.854728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:54.864272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3b784-6ff4-47b9-9fa1-ba2bc703c0d8 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:54.869565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:54.870167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.870439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.870734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.871362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:54.873356 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:54.890364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:54.907578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1014/2027] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:34:54 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:54.911949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:54.922525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15f23a-659c-4b60-ac68-5e4ab13e7ead admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:54.982373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:54.983008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:54.983280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:54.983527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:54.984179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:54.986097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:54.994048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:55.010221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1014/2028] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:34:54 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:55.014639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.025608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fca65-f094-4132-a60c-e08fe8eddc1a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.030719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.031341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.031612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.031877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.032557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.034195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.045268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1015/2029] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:34:55.049681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.060098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219aeaf-26d9-4953-b678-fac9750a486c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.065035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.065697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.065974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.066239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.066883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.068514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319652b-7283-41cd-a079-3ff92f5b475f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.079430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1015/2030] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:55.083716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.094030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f872-31ff-4942-a570-8bc7f7c3c425 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.098820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.099477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.099756 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.100026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] PATH_INFO: `/v3/projects/f409887455bd4eeb81677346f21555d9/users/b5c95e0595f7451f8f84c3f51b040447/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.115121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f409887455bd4eeb81677346f21555d9, user_id=b5c95e0595f7451f8f84c3f51b040447, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.117672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.133461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1016/2031] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:34:55 2022] PUT /identity/v3/projects/f409887455bd4eeb81677346f21555d9/users/b5c95e0595f7451f8f84c3f51b040447/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:55.144482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.144903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.145240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.173643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] MFA Rules not processed for user `b5c95e0595f7451f8f84c3f51b040447`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:55.174930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:55.244057 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1016/2032] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:55.255788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.257588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:55.339767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6312df-4b52-4778-bd78-b292810a0c3c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.347433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.348544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.349032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.349512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.361856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.363424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0eda-a314-428c-9d8a-89e0d1b17752 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.373542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1017/2033] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:55.588564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.663343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eceac7-c12c-48ad-afdf-75b21beb0485 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.669682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.670313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.670584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.670851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] PATH_INFO: `/v3/users/b5c95e0595f7451f8f84c3f51b040447` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.672109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c95e0595f7451f8f84c3f51b040447)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.674155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.731496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c95e0595f7451f8f84c3f51b040447'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:55.738885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c95e0595f7451f8f84c3f51b040447'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:55.750133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:55.752853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c95e0595f7451f8f84c3f51b040447'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:55.753925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c95e0595f7451f8f84c3f51b040447'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:55.765342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c95e0595f7451f8f84c3f51b040447'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:55.771512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1017/2034] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:34:55 2022] DELETE /identity/v3/users/b5c95e0595f7451f8f84c3f51b040447 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:55.776325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.776820 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.776934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.796575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:55.797795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:55.858444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1018/2035] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:55.868306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:55.869937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:55.926275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7137e-c4cd-4741-8578-c741bd65f724 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:55.930115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:55.930720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:55.930991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:55.931256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:55.940149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:55.941612 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b970d9-d1bb-45e2-bd08-3137fa0da5ba service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:55.950820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1018/2036] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:56.231428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.232985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:56.245796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915b1c6-d1af-4bd3-968a-029b367e2da9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.250950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.251604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.251876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.252148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] PATH_INFO: `/v3/projects/f409887455bd4eeb81677346f21555d9` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.253462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f409887455bd4eeb81677346f21555d9)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.255170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.298370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:56.305296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f409887455bd4eeb81677346f21555d9', 'request_id': 'req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:56.312660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f409887455bd4eeb81677346f21555d9', 'request_id': 'req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:56.321522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f409887455bd4eeb81677346f21555d9', 'request_id': 'req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:56.332249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] The token cache is being invalidate because project f409887455bd4eeb81677346f21555d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:34:56.332531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:56.333148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ff263cf1-1c2b-4eb7-9759-d71baa5c79d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:34:56.334626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1019/2037] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:34:56 2022] DELETE /identity/v3/projects/f409887455bd4eeb81677346f21555d9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:56.340611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.341067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.341160 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.361379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:56.362562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:56.423208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1019/2038] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:56.431777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.432063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.432369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.457687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:56.458912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:56.468463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1020/2039] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:34:56.472939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.483386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65030ba8-ce7d-45ec-a7f7-a4dccaa95e76 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.488967 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.489568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.489840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.490103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.490774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:34:56.491068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.492696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.504749 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1020/2040] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:34:56.509341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.519351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65b26-7847-49dc-beaf-2a958272a550 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.524110 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.524745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.525020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.525288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.525946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.527817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.556929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1021/2041] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:56.561209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.571269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7c9e2-a7d2-4237-8d16-9dbdf69b7f18 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.632470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.633084 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.633356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.633619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.634241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.636224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.660977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1021/2042] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:34:56.665283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.676038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d82de-5a3d-455c-b782-8db383e5d215 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.681962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.682586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.682872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.683163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.683814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.685460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.695986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1022/2043] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:56.699992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.710279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80756d6c-dc63-454f-80e0-14fa51ec5262 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.715562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.716266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.716633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.716945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.717596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.719176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b210870-da60-4ded-9f14-b54efb5ce643 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.720503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:56.731378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1022/2044] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:56.735754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.746054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2740d-e0ae-46ce-9b14-ea3ca938a70e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.751314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.751966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.752275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.752549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] PATH_INFO: `/v3/projects/b5daffbc5c3c4628a30f02ce1c3e28e4/users/3cd0bc5ca3364ec8a55e3af3f46ae13c/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.767917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5daffbc5c3c4628a30f02ce1c3e28e4, user_id=3cd0bc5ca3364ec8a55e3af3f46ae13c, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.770619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:56.784714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1023/2045] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:34:56 2022] PUT /identity/v3/projects/b5daffbc5c3c4628a30f02ce1c3e28e4/users/3cd0bc5ca3364ec8a55e3af3f46ae13c/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:34:56.796483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.796769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.797070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.827299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] MFA Rules not processed for user `3cd0bc5ca3364ec8a55e3af3f46ae13c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:56.828541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:56.899938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1023/2046] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:56.904640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:56.975878 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72572a-c74e-4fb9-9a42-043e82a5fa88 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:56.982532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:56.983155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:56.983431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:56.983696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:56.984367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:56.986251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.015626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1024/2047] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:34:56 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:57.019750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.030596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19ebf6-2af9-442c-a728-4c4298f43bfb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.089872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.090520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.090796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.091090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.091728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.093701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.118479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1024/2048] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:34:57 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:57.122579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.133114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854102-5449-46bb-ba25-cd5e8f29347e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.138451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.139105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.139410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.139708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.140442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.142160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.154202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1025/2049] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:57.158252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.173298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933847-048f-41b0-b695-614a368bcd01 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.179878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.180521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.180793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.181053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.181696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.183302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dd320-821d-4407-86bc-9c144d773eff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.194137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1025/2050] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:57.198912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.209928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110f1-2480-4054-b5e4-09a878d212a0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.215761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.216433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.216707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.216971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] PATH_INFO: `/v3/projects/f71e253786424d0198285177ec50a40b/users/7b22c8f7481f470484d72505b9e6e45c/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.232321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71e253786424d0198285177ec50a40b, user_id=7b22c8f7481f470484d72505b9e6e45c, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.234850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.253052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1026/2051] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:34:57 2022] PUT /identity/v3/projects/f71e253786424d0198285177ec50a40b/users/7b22c8f7481f470484d72505b9e6e45c/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:57.263873 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.264142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.264458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.292618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] MFA Rules not processed for user `7b22c8f7481f470484d72505b9e6e45c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:57.293826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:57.351892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:57.367201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1026/2052] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:57.374013 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.395493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:57.442336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aae4-6020-4404-b145-733dd62cce70 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.447180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.447791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.448069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.448370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.449065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.450956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.480616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1027/2053] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:34:57 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:34:57.485461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.497229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f08eea-0bc9-430c-8f2e-df3d4c724d5a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.554574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.555238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.555515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.555786 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.556463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.558549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.581447 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1027/2054] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:34:57 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:34:57.585491 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.596417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae83754-a33c-42c6-943e-83e78c3fc022 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.601647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.602254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.602524 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.602818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.603461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.605091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.615918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1028/2055] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:34:57.620511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.631219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfc689-fc0e-4678-b232-472f39a811d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.635809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.636531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.636829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.637388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] PATH_INFO: `/v3/projects/e9af333b6749461fa558aba57861ffc3/users/c32f6bebfdc645ffbdfbc88a624cce16/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.653162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9af333b6749461fa558aba57861ffc3, user_id=c32f6bebfdc645ffbdfbc88a624cce16, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.655839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.671664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1028/2056] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:34:57 2022] PUT /identity/v3/projects/e9af333b6749461fa558aba57861ffc3/users/c32f6bebfdc645ffbdfbc88a624cce16/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:57.675805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.686034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae13ac-5d52-44a6-9950-78f8d0ca136b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.744049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.744630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.744902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.745147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.745812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.747428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654bb4ed-6abd-436b-803c-20bdb589abf7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.757473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1029/2057] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:34:57.761843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.773076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ceba4d-233f-475f-b177-23ee903fa044 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.778102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.778789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.779065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.779336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] PATH_INFO: `/v3/projects/e9af333b6749461fa558aba57861ffc3/users/c32f6bebfdc645ffbdfbc88a624cce16/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.781286 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9af333b6749461fa558aba57861ffc3, user_id=c32f6bebfdc645ffbdfbc88a624cce16, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:57.783948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:57.794329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1029/2058] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:34:57 2022] PUT /identity/v3/projects/e9af333b6749461fa558aba57861ffc3/users/c32f6bebfdc645ffbdfbc88a624cce16/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:34:57.804487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.804995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.804995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:57.833515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] MFA Rules not processed for user `c32f6bebfdc645ffbdfbc88a624cce16`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:34:57.834739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:57.904071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1030/2059] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:34:57.915588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:57.917554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:57.977036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84b41-c906-4d01-8a33-5d66b11adfdb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:57.981701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:57.982316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:57.982581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:57.982849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:58.041259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:58.042675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:58.047249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:58.052301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1030/2060] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:34:59.583387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:34:59.584857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:34:59.597081 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09ce4d-adbc-44c5-bec3-86a339b4cd7a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:34:59.600876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:34:59.601481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:34:59.601751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:34:59.602014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:34:59.610727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:34:59.658958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:34:59.660397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:34:59.669364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1031/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:34:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:00.234118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:00.242855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879e31d-b5b2-4539-8f48-dde272b51963 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:00.246766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:00.247364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:00.247641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:00.247910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:00.256893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:00.258319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be4438-dac3-461c-8b75-699275f7fd0f service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:00.267932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1031/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:04.225097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:04.290455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c80b-2050-4d90-9a4b-152d58213988 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:04.295830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:04.296481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:04.296752 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:04.297018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] PATH_INFO: `/v3/users/3cd0bc5ca3364ec8a55e3af3f46ae13c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:04.298235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd0bc5ca3364ec8a55e3af3f46ae13c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:04.300147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:04.347873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd0bc5ca3364ec8a55e3af3f46ae13c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.348580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd0bc5ca3364ec8a55e3af3f46ae13c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.360578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd0bc5ca3364ec8a55e3af3f46ae13c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.369517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd0bc5ca3364ec8a55e3af3f46ae13c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.370141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd0bc5ca3364ec8a55e3af3f46ae13c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.382288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1032/2063] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:35:04 2022] DELETE /identity/v3/users/3cd0bc5ca3364ec8a55e3af3f46ae13c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:04.386922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:04.387722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:04.387837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:04.406564 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:04.407639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:04.465504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1032/2064] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:04.475886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:04.477689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:04.511657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:04.541588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf82e-5d07-49de-829b-9432fa054fae admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:04.545146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:04.545754 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:04.546028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:04.546293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:04.555330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:04.556775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:04.567098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1033/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:04.856621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:04.858075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:04.870269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c5dc-e197-4731-9e17-bce8f0832a81 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:04.875992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:04.876672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:04.876957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:04.877205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] PATH_INFO: `/v3/projects/b5daffbc5c3c4628a30f02ce1c3e28e4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:04.878436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5daffbc5c3c4628a30f02ce1c3e28e4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:04.880494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:04.931104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5daffbc5c3c4628a30f02ce1c3e28e4', 'request_id': 'req-b8717c4a-331a-4ce1-9ae5-117e7eee907f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.937775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5daffbc5c3c4628a30f02ce1c3e28e4', 'request_id': 'req-b8717c4a-331a-4ce1-9ae5-117e7eee907f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.946462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5daffbc5c3c4628a30f02ce1c3e28e4', 'request_id': 'req-b8717c4a-331a-4ce1-9ae5-117e7eee907f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.956994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] The token cache is being invalidate because project b5daffbc5c3c4628a30f02ce1c3e28e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:35:04.957343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.957965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:04.959470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1033/2066] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:35:04 2022] DELETE /identity/v3/projects/b5daffbc5c3c4628a30f02ce1c3e28e4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:04.963662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:04.973703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66572b6e-7dfe-41b4-ac0d-4b9e8f46b79a service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:05.032238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:05.032868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:05.033143 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:05.033408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] PATH_INFO: `/v3/users/7b22c8f7481f470484d72505b9e6e45c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:05.034539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b22c8f7481f470484d72505b9e6e45c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:05.036471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:05.085176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b22c8f7481f470484d72505b9e6e45c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.085815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b22c8f7481f470484d72505b9e6e45c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.098444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b22c8f7481f470484d72505b9e6e45c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.107442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b22c8f7481f470484d72505b9e6e45c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.108077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b22c8f7481f470484d72505b9e6e45c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.120522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1034/2067] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:35:04 2022] DELETE /identity/v3/users/7b22c8f7481f470484d72505b9e6e45c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:05.399591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:05.401122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:05.424527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717c4a-331a-4ce1-9ae5-117e7eee907f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:05.486438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:05.486991 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:05.487295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:05.487572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] PATH_INFO: `/v3/projects/f71e253786424d0198285177ec50a40b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:05.488719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71e253786424d0198285177ec50a40b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:05.490637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:05.502022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:05.537929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71e253786424d0198285177ec50a40b', 'request_id': 'req-2de35373-33f8-44a3-b045-278039cdab21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.544100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71e253786424d0198285177ec50a40b', 'request_id': 'req-2de35373-33f8-44a3-b045-278039cdab21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.554201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71e253786424d0198285177ec50a40b', 'request_id': 'req-2de35373-33f8-44a3-b045-278039cdab21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.565060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] The token cache is being invalidate because project f71e253786424d0198285177ec50a40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:35:05.565339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.565954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.567510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1034/2068] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:35:05 2022] DELETE /identity/v3/projects/f71e253786424d0198285177ec50a40b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:05.571872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:05.573414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:05.586517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2b80-0bd4-470e-8268-ddab023081dc admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:05.634464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:05.647369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:05.648328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:05.648602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:05.648870 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] PATH_INFO: `/v3/users/c32f6bebfdc645ffbdfbc88a624cce16` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:05.650009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32f6bebfdc645ffbdfbc88a624cce16)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:05.651955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:05.699254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32f6bebfdc645ffbdfbc88a624cce16'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.699911 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32f6bebfdc645ffbdfbc88a624cce16'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.711920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32f6bebfdc645ffbdfbc88a624cce16'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.721171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32f6bebfdc645ffbdfbc88a624cce16'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.721782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-8820cfcf-718a-4797-afb8-b85a14667343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32f6bebfdc645ffbdfbc88a624cce16'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:05.734332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1035/2069] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:35:05 2022] DELETE /identity/v3/users/c32f6bebfdc645ffbdfbc88a624cce16 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:06.018138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.019570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:06.034796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35373-33f8-44a3-b045-278039cdab21 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.096351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.096981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.097256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.097523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] PATH_INFO: `/v3/projects/e9af333b6749461fa558aba57861ffc3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.098793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9af333b6749461fa558aba57861ffc3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.100818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.150228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9af333b6749461fa558aba57861ffc3', 'request_id': 'req-e4b62c27-475b-42e2-b7d9-68c9003855f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:06.158389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9af333b6749461fa558aba57861ffc3', 'request_id': 'req-e4b62c27-475b-42e2-b7d9-68c9003855f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:06.166529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9af333b6749461fa558aba57861ffc3', 'request_id': 'req-e4b62c27-475b-42e2-b7d9-68c9003855f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:06.176840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] The token cache is being invalidate because project e9af333b6749461fa558aba57861ffc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:35:06.177103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:06.177800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e4b62c27-475b-42e2-b7d9-68c9003855f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:06.179351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1035/2070] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:35:06 2022] DELETE /identity/v3/projects/e9af333b6749461fa558aba57861ffc3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:06.185239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.185521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.185787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.206511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:06.207762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:06.267893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1036/2071] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:06.279047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.279341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.279613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.299374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:06.300604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:06.309823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1036/2072] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:35:06.314114 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.323910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af4da6-0ae2-4e89-8071-9a82086e0dcd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.330323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.330923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.331195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.331445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.332109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:35:06.332457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.334005 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.345788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1037/2073] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:35:06.350098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.359562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614af821-b482-49d4-bf7e-297d40182bc7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.364610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.365195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.365465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.365730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.366356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.368305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.397971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1037/2074] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:35:06 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:06.402601 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.415489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e0e70-0b75-4f81-a099-15ae55b6fdc2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.467215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:06.484865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.485714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.486003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.486523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.487205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.489932 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.514289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1038/2075] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:35:06 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:35:06.518990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.526394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:06.536419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e1b7e-6f83-4833-936b-7d3dc4c56c43 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.543078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.543661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.543940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.544242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.544924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.546640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.559353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1038/2076] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:06.563893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.574877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f8e80-d69f-4c30-b0de-bfaf174ded93 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.580863 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.581650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.581925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.582281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.582934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.584556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38f550-a9fb-4667-b527-11bfdac9d308 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.596010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1039/2077] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:06.600538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.611021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355efa66-d905-4e88-bd92-cf386609ff11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.616362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.617018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.617293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.617558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] PATH_INFO: `/v3/projects/a97dd511f689433182caa96bb3382fb5/users/7c207cac8cb34c28a8854df3d1d4b459/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.633823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97dd511f689433182caa96bb3382fb5, user_id=7c207cac8cb34c28a8854df3d1d4b459, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.636523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.651860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1039/2078] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:35:06 2022] PUT /identity/v3/projects/a97dd511f689433182caa96bb3382fb5/users/7c207cac8cb34c28a8854df3d1d4b459/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:06.662427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.662716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.663014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.692043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] MFA Rules not processed for user `7c207cac8cb34c28a8854df3d1d4b459`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:06.693282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:06.774767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1040/2079] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:06.786796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:06.788868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:06.853975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121b05a-812c-44a8-a6d9-be9183dbef25 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:06.858799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f056f1106e9a4b5ba29fe99058e47c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:06.859405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:06.859642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:06.859950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:06.869619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:06.871065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663009de-1c10-49a2-bc6c-dcf28a850e97 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:06.880607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1040/2080] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:14.966041 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:14.967613 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:15.107961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:15.114391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52193c-fb8c-44d9-9db6-5ac7fb2d5827 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:15.120074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:15.120706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:15.120981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:15.121248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:15.123031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:15.135582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:15.137521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:15.147273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1041/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:16.875924 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:25.143128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:30.576355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f1ee-aafc-4cfd-821a-a98ae8cd090c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:30.576893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f1ee-aafc-4cfd-821a-a98ae8cd090c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:30.576893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f1ee-aafc-4cfd-821a-a98ae8cd090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:30.596804 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5a47f1ee-aafc-4cfd-821a-a98ae8cd090c None None] MFA Rules not processed for user `9a4a84c1ffc4466687122a6c67536d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:30.598028 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a47f1ee-aafc-4cfd-821a-a98ae8cd090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:30.654724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1041/2082] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:40.647880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=25 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:42.280925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:42.370619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc77e-c352-46c5-be93-34ed5323c8a5 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:42.375939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:42.376605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:42.376915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:42.377180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] PATH_INFO: `/v3/users/7c207cac8cb34c28a8854df3d1d4b459` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:42.378441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c207cac8cb34c28a8854df3d1d4b459)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:42.380385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:42.432506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c207cac8cb34c28a8854df3d1d4b459'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:42.433148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c207cac8cb34c28a8854df3d1d4b459'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:42.445160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c207cac8cb34c28a8854df3d1d4b459'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:42.453561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c207cac8cb34c28a8854df3d1d4b459'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:42.454255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c207cac8cb34c28a8854df3d1d4b459'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:42.467174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1042/2083] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:35:42 2022] DELETE /identity/v3/users/7c207cac8cb34c28a8854df3d1d4b459 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:42.471871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:42.472487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:42.472487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:42.513324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:42.514506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:42.574819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1042/2084] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:42.584784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:42.586289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:42.659615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:42.670991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb7ee-e6af-4741-b5fe-d0586524ab8d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:42.675206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:42.675881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:42.676189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:42.676467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:42.686093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:42.687645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403edb2d-ab1e-44b0-927a-451f7b55a1c0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:42.698262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1043/2085] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:42.987212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:42.988837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:43.001140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413680-8934-4b3d-bdb7-b255a65874c6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.006278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.006915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.007193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.007465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] PATH_INFO: `/v3/projects/a97dd511f689433182caa96bb3382fb5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.008602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97dd511f689433182caa96bb3382fb5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.010545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.062843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97dd511f689433182caa96bb3382fb5', 'request_id': 'req-f7623ce0-2304-491a-98b6-56d27c032e26'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.069464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97dd511f689433182caa96bb3382fb5', 'request_id': 'req-f7623ce0-2304-491a-98b6-56d27c032e26'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.077689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97dd511f689433182caa96bb3382fb5', 'request_id': 'req-f7623ce0-2304-491a-98b6-56d27c032e26'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.088137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] The token cache is being invalidate because project a97dd511f689433182caa96bb3382fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:35:43.088540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.089152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7623ce0-2304-491a-98b6-56d27c032e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.090693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1043/2086] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:35:42 2022] DELETE /identity/v3/projects/a97dd511f689433182caa96bb3382fb5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:43.097361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.097644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.097912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.159014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:43.160796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:43.226904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1044/2087] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:35:43.234777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.235060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.235327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.255414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:43.256720 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:43.265208 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1044/2088] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:43.269690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.279876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb931c4-cd51-428f-bb04-f28cfcd5d1c2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.285587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.286396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.287436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.287730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.288573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:35:43.288990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.290691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.303285 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1045/2089] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:35:43.308314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.317826 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633910d-103a-4fd3-a915-c7a278b3c453 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.323009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.323626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.323906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.324192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.324817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.326864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.355022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1045/2090] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:35:43 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:43.359359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.369403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416787b-aa7a-430b-904f-87b47e198048 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.429470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.430030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.430354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.430634 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.431272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.433303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.455769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1046/2091] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:35:43 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:43.460186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.470509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ee309-f01a-439e-97ad-ff7dcf2c75b7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.475269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.475976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.476290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.476560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.477261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.478830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.489236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1046/2092] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:35:43.493418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.503718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1e50-3d25-4650-882b-9cc2e589329a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.508882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.509499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.509770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.510035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.510692 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.512659 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52705faf-06d1-4890-be49-9a6b83e5a282 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.528144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1047/2093] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:43.529614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.541915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac417-35d2-472f-bc12-9325eaf8f1a5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.547064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.547785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.548092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.555782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] PATH_INFO: `/v3/projects/b6d4c167a6c441f2a5f658251e11640c/users/d934498bd982469cbc4e4716766b7118/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.575592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d4c167a6c441f2a5f658251e11640c, user_id=d934498bd982469cbc4e4716766b7118, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.578359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.607802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1047/2094] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:35:43 2022] PUT /identity/v3/projects/b6d4c167a6c441f2a5f658251e11640c/users/d934498bd982469cbc4e4716766b7118/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:43.610844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.611704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.612029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.645370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] MFA Rules not processed for user `d934498bd982469cbc4e4716766b7118`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:43.646547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:43.659747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:43.714277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1048/2095] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:43.726293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.728636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:43.732863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:43.816157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d397a-f9dd-4d14-9282-cc5de82277c2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.820362 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.821204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.821204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.821476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.830542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.832098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f10f9-ead5-4e9d-bd63-b3e74ed5b3a7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.842462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1048/2096] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:43.868129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:43.932718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e879e-a529-48fb-b6fa-128cf7f19b55 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:43.937445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:43.938089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:43.938368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:43.938642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] PATH_INFO: `/v3/users/d934498bd982469cbc4e4716766b7118` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:43.939796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d934498bd982469cbc4e4716766b7118)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:43.941777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:43.990480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd934498bd982469cbc4e4716766b7118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:43.991064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd934498bd982469cbc4e4716766b7118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.002848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd934498bd982469cbc4e4716766b7118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.010749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd934498bd982469cbc4e4716766b7118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.011876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd934498bd982469cbc4e4716766b7118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.025021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1049/2097] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:35:43 2022] DELETE /identity/v3/users/d934498bd982469cbc4e4716766b7118 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:44.029752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.030024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.030296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.049661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:44.050864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:44.112285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1049/2098] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:44.122359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.123898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:44.164284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:44.189798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d6f10-b6b3-4946-a4e3-8e6d52da004f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.195129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.195740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.196014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.196311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.206557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.208898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5e142-1a59-473d-ab52-7cba5b6e847d service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:44.224259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1050/2099] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:44.524435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.526089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:44.540431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901097f-0148-4f9d-b200-cf9531b00499 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.545716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.546777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.546986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.547270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] PATH_INFO: `/v3/projects/b6d4c167a6c441f2a5f658251e11640c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.548434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d4c167a6c441f2a5f658251e11640c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.550376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:44.599398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d4c167a6c441f2a5f658251e11640c', 'request_id': 'req-375b2d46-e823-4cf4-b57a-4aff2fc3d412'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.605758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d4c167a6c441f2a5f658251e11640c', 'request_id': 'req-375b2d46-e823-4cf4-b57a-4aff2fc3d412'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.614844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d4c167a6c441f2a5f658251e11640c', 'request_id': 'req-375b2d46-e823-4cf4-b57a-4aff2fc3d412'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.625068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] The token cache is being invalidate because project b6d4c167a6c441f2a5f658251e11640c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:35:44.625347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.625992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-375b2d46-e823-4cf4-b57a-4aff2fc3d412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:35:44.627524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1050/2100] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:35:44 2022] DELETE /identity/v3/projects/b6d4c167a6c441f2a5f658251e11640c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:44.632750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.633254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.633353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.656368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:44.657655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:44.726495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1051/2101] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:35:44.734305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.734587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.734856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.754910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:44.756099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:44.765223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1051/2102] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:44.770079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.785174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c251876-f376-48ec-9621-efe8896f3267 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.792173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.792730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.793001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.793262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.793912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:35:44.794209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.795784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:44.809838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1052/2103] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:35:44.813592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.818373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:44.823899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515de8-6ef2-4349-bad6-7157dbcc8c16 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.831763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.832724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.833220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.833734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.834407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.836404 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:44.867099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1052/2104] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:35:44 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:44.873234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.886757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf1d02-862d-4164-b69c-bfe970f5a306 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.945720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.946330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.946605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.946867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.947487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.949427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:44.973347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1053/2105] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:35:44 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:44.977651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:44.988636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c3cc-ed1f-48fd-95fb-7757cbc0d9c2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:44.994182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:44.994785 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:44.995056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:44.995326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:44.995962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:44.997628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:45.007866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1053/2106] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:45.012483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:45.022573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac2287-67b0-4b78-a775-c4f92cec0bd3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:45.027241 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:45.027845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:45.028138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:45.028411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:45.029059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:45.030750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2534dd4-aecc-4489-a4a2-4ef48d7db517 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:45.041970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1054/2107] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:45.046115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:45.057281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e06d0-4de7-47b8-b5ef-6efa76fc6615 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:45.062983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:45.063652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:45.063930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:45.064242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] PATH_INFO: `/v3/projects/81282f5bd6b34d90afaba281d5bbc16b/users/983c742ffffe4a82a722eb1b1c23f971/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:45.081666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81282f5bd6b34d90afaba281d5bbc16b, user_id=983c742ffffe4a82a722eb1b1c23f971, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:45.084428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:45.098719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1054/2108] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:35:45 2022] PUT /identity/v3/projects/81282f5bd6b34d90afaba281d5bbc16b/users/983c742ffffe4a82a722eb1b1c23f971/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:45.103873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c7bb-0034-450f-b65f-99fe3d396e2e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:45.104237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c7bb-0034-450f-b65f-99fe3d396e2e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:45.104509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c7bb-0034-450f-b65f-99fe3d396e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:45.123872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-17b8c7bb-0034-450f-b65f-99fe3d396e2e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:45.125148 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-17b8c7bb-0034-450f-b65f-99fe3d396e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:45.166354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:45.186774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1055/2109] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:45.198479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:45.200298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:45.257969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee652b-cddb-4066-84b1-bca430b3dc1a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:45.262418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:45.263103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:45.263501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:45.263772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:45.274231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:45.275712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:45.286209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1055/2110] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:51.086990 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.087579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.087579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.148876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] MFA Rules not processed for user `983c742ffffe4a82a722eb1b1c23f971`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:51.150619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:51.249022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1056/2111] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:51.250161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.260889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dfbbc-86d5-4751-91f5-071d9035f5a6 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.266038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.266699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.266976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.267269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.267935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.269964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.301710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1056/2112] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:35:51 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:35:51.305957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.318727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cca80-018c-4a1d-b41b-b1ad4bf11cd3 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.385068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.385701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.385977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.386249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.387520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.389617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.414379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1057/2113] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:35:51 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:35:51.418915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.430982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c9f1f-966e-4fa4-9f81-811f6906e105 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.436232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.436867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.437143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.437408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.438070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.439694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.451473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1057/2114] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:51.455797 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.466444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80d1b1-32cc-4698-9cc3-cc7d9d373ba5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.471235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.472004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.472214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.472525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] PATH_INFO: `/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742/users/41e18bc015b545acaa92588c84622a5a/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.488931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0e2b1a36404275a8e53fbbd15b6742, user_id=41e18bc015b545acaa92588c84622a5a, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.491651 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.509400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1058/2115] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:35:51 2022] PUT /identity/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742/users/41e18bc015b545acaa92588c84622a5a/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:51.510345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.521907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc370-866d-48f8-a158-bdd8dc78715f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.584806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.585821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.586102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.586517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.587243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.588914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.603415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1058/2116] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:35:51.604723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:51.615973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713337e-94f5-4dc2-8910-d942b61250a2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:51.620864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:51.621541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:51.621818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:51.622087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] PATH_INFO: `/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742/users/41e18bc015b545acaa92588c84622a5a/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:51.623999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0e2b1a36404275a8e53fbbd15b6742, user_id=41e18bc015b545acaa92588c84622a5a, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:51.626741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f04d68-4acd-4533-aadc-ea6653c6831b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:51.638560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1059/2117] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:35:51 2022] PUT /identity/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742/users/41e18bc015b545acaa92588c84622a5a/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:35:54.183196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:54.185171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:54.204347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:54.369073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442b26-b501-4f8b-bb16-f34b3fb69cc7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:54.465997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:54.466822 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:54.467190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:54.467551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:54.470694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:54.580481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:54.582654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:54.611805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1059/2118] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:56.105949 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476e50-3483-40b4-b3ab-f50944857c6d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:56.106440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476e50-3483-40b4-b3ab-f50944857c6d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:56.106902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd476e50-3483-40b4-b3ab-f50944857c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:56.147200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-bd476e50-3483-40b4-b3ab-f50944857c6d None None] MFA Rules not processed for user `41e18bc015b545acaa92588c84622a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:56.148640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bd476e50-3483-40b4-b3ab-f50944857c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:56.177391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:56.247649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1060/2119] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:35:56.256395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:35:56.276642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2702b-cc52-4e75-ab9b-d59516010bb9 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:35:56.285278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:35:56.286073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:56.286194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:56.286436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:56.305392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:35:56.366324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:35:56.368342 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34394ac6-6a2c-4b17-8fba-29790055e0b8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:35:56.377931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1060/2120] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:35:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:35:57.560472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:35:57.560979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:35:57.561078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:35:57.603245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] MFA Rules not processed for user `983c742ffffe4a82a722eb1b1c23f971`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:35:57.604927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:35:57.612602 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1061/2121] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:36:03.213807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:03.214383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:03.214643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:03.246849 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:36:03.248042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:03.304938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1061/2122] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:36:03.355151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:03.356620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:03.434100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01f7b5-c726-4013-a3cb-9f7593b9fc16 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:03.439609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:03.440258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:03.440545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:03.440818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:03.450907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:03.452530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:03.461132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1062/2123] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:36:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:36:03.829539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:03.891239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091935a-0f5e-4952-a736-8e19d0e67a89 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:03.895287 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:03.895902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:03.896206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:03.896471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:03.898482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:03.965568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:03.966964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:03.977737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1062/2124] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:36:07.310232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:07.313312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:07.402471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd508fc-73b9-4569-99f4-419e38c505c9 service swift] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:07.410064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:07.411231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:07.411765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:07.412380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:07.414468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:07.429679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:07.431481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:07.450654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1063/2125] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:36:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:36:10.361272 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:10.362809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:10.371171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:10.375210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f60e-2213-497d-b746-89efec93bf91 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:10.380989 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:10.381688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:10.381960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:10.382223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] PATH_INFO: `/v3/users/ee084e4a27a94d409653f36e786af9e7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:10.383346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee084e4a27a94d409653f36e786af9e7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:10.385411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:10.435227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee084e4a27a94d409653f36e786af9e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.442428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee084e4a27a94d409653f36e786af9e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.454837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee084e4a27a94d409653f36e786af9e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.455534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee084e4a27a94d409653f36e786af9e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.464862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee084e4a27a94d409653f36e786af9e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.471101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1063/2126] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:36:10 2022] DELETE /identity/v3/users/ee084e4a27a94d409653f36e786af9e7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:36:10.654243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:10.665119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570b30-1844-4563-a566-6c8499dc8830 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:10.685814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:10.722847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:10.723668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:10.723872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:10.724145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] PATH_INFO: `/v3/projects/07b1cc93b68340b787ee0c99f1bf7c68` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:10.725328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b1cc93b68340b787ee0c99f1bf7c68)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:10.727319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:10.779818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b1cc93b68340b787ee0c99f1bf7c68', 'request_id': 'req-14321cb2-5a2a-45ba-b964-4748893606e0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.786361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b1cc93b68340b787ee0c99f1bf7c68', 'request_id': 'req-14321cb2-5a2a-45ba-b964-4748893606e0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.795302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b1cc93b68340b787ee0c99f1bf7c68', 'request_id': 'req-14321cb2-5a2a-45ba-b964-4748893606e0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.806269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] The token cache is being invalidate because project 07b1cc93b68340b787ee0c99f1bf7c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:36:10.806556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.807169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.808715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1064/2127] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:36:10 2022] DELETE /identity/v3/projects/07b1cc93b68340b787ee0c99f1bf7c68 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:36:10.812797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:10.814245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:10.827116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae3384-3ec1-4610-8379-f3cb0ef65fba admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:10.895430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:10.896076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:10.896386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:10.896655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] PATH_INFO: `/v3/users/e00aa4c96de546ac809c55947f427488` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:10.897789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00aa4c96de546ac809c55947f427488)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:10.899726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:10.948392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00aa4c96de546ac809c55947f427488'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.955149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00aa4c96de546ac809c55947f427488'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.966653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00aa4c96de546ac809c55947f427488'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.967295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00aa4c96de546ac809c55947f427488'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.971037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:10.976974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08cb0f52-2152-45d5-9c07-0325a53cd7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00aa4c96de546ac809c55947f427488'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:10.983676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1064/2128] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:36:10 2022] DELETE /identity/v3/users/e00aa4c96de546ac809c55947f427488 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:36:11.162244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:11.164154 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:11.181443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321cb2-5a2a-45ba-b964-4748893606e0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:11.240238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:11.240904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:11.241180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:11.241445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] PATH_INFO: `/v3/projects/be0e7e9dc2e741b3922a0200f9786605` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:11.242558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0e7e9dc2e741b3922a0200f9786605)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:11.244484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:11.293667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0e7e9dc2e741b3922a0200f9786605', 'request_id': 'req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:11.299801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0e7e9dc2e741b3922a0200f9786605', 'request_id': 'req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:11.302784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:11.309537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0e7e9dc2e741b3922a0200f9786605', 'request_id': 'req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:11.321563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] The token cache is being invalidate because project be0e7e9dc2e741b3922a0200f9786605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:36:11.321851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:11.322479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:36:11.323982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1065/2129] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:36:11 2022] DELETE /identity/v3/projects/be0e7e9dc2e741b3922a0200f9786605 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:36:20.982910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:21.318518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=3,keystone:SELECT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:38.540889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:38.541397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:38.541397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:38.561395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:36:38.562599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:38.621515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1065/2130] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:36:38.633655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:38.635079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:38.703676 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00aa8a-125e-4b1f-8b80-2fe3fe572d84 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:38.710155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:38.710777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:38.711050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:38.711316 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:38.721410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:38.723093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a28ff-53cc-4abe-bc11-a67cf3f0b848 service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:38.731780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1066/2131] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:36:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:36:41.607867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:41.609360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:41.676088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9969b3-075b-4390-a535-12d1ffa77c18 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:41.679766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:41.680436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:41.680717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:41.680985 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:41.683013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:41.776717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:41.778194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ac100-8158-45fb-a73f-2e086cd10d46 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:41.789906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1066/2132] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:36:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:36:48.728470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=32 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:51.784735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=75 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:36:53.631215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:53.631813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:53.631813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:53.653069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] MFA Rules not processed for user `2fc3a4b9afd14409b3aef17f49aafd29`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:36:53.654501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:53.714128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1067/2133] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:36:55.802291 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:55.802795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:55.802907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:55.822299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:36:55.823501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:36:55.830908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1067/2134] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:36:55.843828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:36:55.858246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd601f69-00b1-4208-a040-11621c96f284 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:36:55.863349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:36:55.863976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:36:55.864290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:36:55.864566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:36:55.873062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:36:55.874667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:36:55.883135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1068/2135] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:36:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:37:05.816648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:05.879713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=31 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:05.972614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:05.982796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cfbe8-9eec-4135-9a21-87d87890497f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:05.987099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:05.987715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:05.987986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:05.988292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:05.990156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:06.068568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:06.070011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:06.081095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1068/2136] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:08.893585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:08.903923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d145a7-0796-44a0-8b17-79a8012055f2 service swift] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:08.907829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:08.908467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:08.908737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:08.908993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:08.910774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:08.924363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:08.925782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bf14e-d674-4c44-a5b4-225082463b99 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:08.936424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1069/2137] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:37:12.090134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:12.100858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e179e8-07b5-4b5c-b7f9-45a87970f7b4 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:12.104529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:12.105166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:12.105442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:12.105713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:12.107543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:12.193364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:12.194946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:12.214595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1069/2138] 10.222.0.15 () {60 vars in 1396 bytes} [Tue Oct 11 13:37:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:17.314582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:17.315160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:17.315160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:17.339701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:17.340989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:17.349076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1070/2139] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:37:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:17.372369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:17.383926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dca57-121a-417f-89c0-8ec6c2a58c90 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:17.389335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:17.389946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:17.390248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:17.390513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:17.398556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:17.400171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:17.408892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1070/2140] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:37:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:37:27.333476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=9 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:27.405755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:44.326794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:44.328618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:44.341225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbed3e8-151f-4c16-bdca-5a7481662758 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:44.345555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:44.346184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:44.346464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:44.346741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:44.348622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:44.418460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:44.419999 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:44.430831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1071/2141] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:47.494543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:47.496930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:47.583625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:47.585521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:47.603130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fcf48-dbf2-4ec7-94dc-62065be74d24 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:47.603805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbdfe-132f-41f0-8b49-dbb4f46c2a15 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:47.609175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:47.609334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:47.609988 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:47.610306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:47.610444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:47.610769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] PATH_INFO: `/v3/users/9a4a84c1ffc4466687122a6c67536d9b` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:47.611027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:47.611536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:47.613684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:47.626398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:47.627998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:47.629629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4a84c1ffc4466687122a6c67536d9b)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:47.631611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:47.638488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1071/2142] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:47.680490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4a84c1ffc4466687122a6c67536d9b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:47.681184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4a84c1ffc4466687122a6c67536d9b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:47.693075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4a84c1ffc4466687122a6c67536d9b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:47.701723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4a84c1ffc4466687122a6c67536d9b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:47.702348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6902328b-9037-4793-b53a-daa858262bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4a84c1ffc4466687122a6c67536d9b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:47.715827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1072/2143] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:37:47 2022] DELETE /identity/v3/users/9a4a84c1ffc4466687122a6c67536d9b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:37:47.908730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:47.919771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5e28-ac8a-4b51-878f-22dff78d1ba1 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:47.984973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:47.985617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:47.985896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:47.986166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] PATH_INFO: `/v3/projects/33ec6c4b270341c5a4ebce407bb5cca9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:47.996922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ec6c4b270341c5a4ebce407bb5cca9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:47.998829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.045601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ec6c4b270341c5a4ebce407bb5cca9', 'request_id': 'req-58db4dd4-9889-4a6c-9dfa-455b15ad5960'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:48.052299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ec6c4b270341c5a4ebce407bb5cca9', 'request_id': 'req-58db4dd4-9889-4a6c-9dfa-455b15ad5960'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:48.060247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ec6c4b270341c5a4ebce407bb5cca9', 'request_id': 'req-58db4dd4-9889-4a6c-9dfa-455b15ad5960'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:48.071526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] The token cache is being invalidate because project 33ec6c4b270341c5a4ebce407bb5cca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:37:48.071850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:48.072546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58db4dd4-9889-4a6c-9dfa-455b15ad5960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:48.074061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1072/2144] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:37:47 2022] DELETE /identity/v3/projects/33ec6c4b270341c5a4ebce407bb5cca9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:48.078970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.079253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.079561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.100028 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:48.101304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:48.151287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:48.168268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1073/2145] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:37:48.173792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.174074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.174371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.194791 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:48.195979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:48.204143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1073/2146] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:48.208348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.218115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b2af0-9246-4e51-8b11-e8da175b6676 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.223047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.223685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.223965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.224267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.224921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:37:48.225199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.226802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.239062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1074/2147] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:37:48.243216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.248859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:48.253371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad742709-008d-4728-b5e1-5e535eb86699 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.260029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.260794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.261450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.261935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.262840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.264949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.306450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1074/2148] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:37:48 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:48.307734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.317822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a3ffc-8c7a-4b7d-9a24-4a1c6d105036 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.373847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.374366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.374638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.374900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.375518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.377448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.400009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1075/2149] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:37:48 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:37:48.404204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.418029 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa227c-e10d-4a96-a24e-fd5d617cfe8e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.426334 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.427187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.428013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.428013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.428969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.430979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.445909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1075/2150] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:48.447830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.459353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c585b83-81ce-4723-841b-461c73e72d61 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.464449 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.465284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.465760 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.466321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.467064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.468784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeac47-2842-4ee7-84f4-000702b44e73 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.482029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1076/2151] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:48.486361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.499660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee83fac-364e-48e3-a15e-1080528e5be0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.504233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.504886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.505161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.505428 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] PATH_INFO: `/v3/projects/bd01ae87f608421d99145292f312cf5b/users/cfe2ce8c1d184aba861a8b37777dc488/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.521115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd01ae87f608421d99145292f312cf5b, user_id=cfe2ce8c1d184aba861a8b37777dc488, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.523760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.539223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1076/2152] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:48 2022] PUT /identity/v3/projects/bd01ae87f608421d99145292f312cf5b/users/cfe2ce8c1d184aba861a8b37777dc488/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:48.545200 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ec44-2950-444a-af3d-d6a00b5a862d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.545617 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ec44-2950-444a-af3d-d6a00b5a862d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.546959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ec44-2950-444a-af3d-d6a00b5a862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.571407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fb70ec44-2950-444a-af3d-d6a00b5a862d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:48.572678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fb70ec44-2950-444a-af3d-d6a00b5a862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:48.637007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1077/2153] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:48.648522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:48.650408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:48.711248 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6174-bd88-4707-9433-9362e5d1d6a5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:48.715400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:48.716013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:48.716315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:48.716581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:48.726619 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:48.728085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:48.738216 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1077/2154] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:49.382718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.383027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.383328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.412359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] MFA Rules not processed for user `cfe2ce8c1d184aba861a8b37777dc488`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:49.413615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:49.463280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:49.482769 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1078/2155] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:37:49.487300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.497143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26657-2f1d-414a-b464-e9aaea83d757 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.501778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.502364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.502640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.502908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.503544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.505490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.534318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1078/2156] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:37:49 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:49.538437 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.549169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9b344-5ad0-411e-a249-f76428815594 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.608973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.609589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.609865 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.610195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.610830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.612812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.637051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1079/2157] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:37:49 2022] POST /identity/v3/users => generated 308 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:37:49.641198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.651953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11399161-c061-4516-a1eb-61a21b43c117 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.657194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.657783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.658061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.658329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.658981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.660625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.671653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1079/2158] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:49.675746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.685964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa85eb-279c-445d-8722-41eef6cbb323 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.691089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.691758 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.692033 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.692331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] PATH_INFO: `/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af/users/7b6b607fa7bd4f4f8c482424569abe34/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.708071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7ace275bfb469ab65ce79fb22cf0af, user_id=7b6b607fa7bd4f4f8c482424569abe34, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.710746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.725185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1080/2159] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:49 2022] PUT /identity/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af/users/7b6b607fa7bd4f4f8c482424569abe34/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:49.729232 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.739671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b7c7-42ee-42d3-8053-b52de6efc425 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.798872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.799470 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.799746 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.800002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.800774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.802381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7c3ba-219f-4991-a090-dc2fbb0dd428 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.812071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1080/2160] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:49.818240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:49.830920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2aa5d-1a5c-43cd-a90d-cbab05e76464 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:49.836167 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:49.836804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:49.837080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:49.837348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] PATH_INFO: `/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af/users/7b6b607fa7bd4f4f8c482424569abe34/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:49.839312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7ace275bfb469ab65ce79fb22cf0af, user_id=7b6b607fa7bd4f4f8c482424569abe34, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:49.841964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:49.853129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1081/2161] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:49 2022] PUT /identity/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af/users/7b6b607fa7bd4f4f8c482424569abe34/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:50.511965 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:50.512614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:50.512614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:50.520921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:50.542705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] MFA Rules not processed for user `7b6b607fa7bd4f4f8c482424569abe34`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:50.544229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:50.616823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1081/2162] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:51.646055 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:51.710662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ba07-3db1-4e17-a09c-ef9b2591e630 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:51.716086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:51.716731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:51.717010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:51.717280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] PATH_INFO: `/v3/users/cfe2ce8c1d184aba861a8b37777dc488` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:51.718433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe2ce8c1d184aba861a8b37777dc488)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:51.720371 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:51.767273 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:51.769069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe2ce8c1d184aba861a8b37777dc488'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:51.769931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe2ce8c1d184aba861a8b37777dc488'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:51.782746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe2ce8c1d184aba861a8b37777dc488'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:51.790076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe2ce8c1d184aba861a8b37777dc488'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:51.790681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe2ce8c1d184aba861a8b37777dc488'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:51.802945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1082/2163] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:37:51 2022] DELETE /identity/v3/users/cfe2ce8c1d184aba861a8b37777dc488 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:51.989237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.000783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170cc42-e93e-43c4-9275-877ed3296308 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.059678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.060364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.060639 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.060905 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] PATH_INFO: `/v3/projects/bd01ae87f608421d99145292f312cf5b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.062059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd01ae87f608421d99145292f312cf5b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.064000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.112027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd01ae87f608421d99145292f312cf5b', 'request_id': 'req-ddde6e80-42ef-4609-90c1-1f79a2ffd232'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.118374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd01ae87f608421d99145292f312cf5b', 'request_id': 'req-ddde6e80-42ef-4609-90c1-1f79a2ffd232'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.128490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd01ae87f608421d99145292f312cf5b', 'request_id': 'req-ddde6e80-42ef-4609-90c1-1f79a2ffd232'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.139295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] The token cache is being invalidate because project bd01ae87f608421d99145292f312cf5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:37:52.139577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.140348 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.141909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1082/2164] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:37:51 2022] DELETE /identity/v3/projects/bd01ae87f608421d99145292f312cf5b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:52.146092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.147612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:52.161461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77869f9-cc10-408e-847e-4f00084d3034 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.219805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.220469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.220743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.221014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] PATH_INFO: `/v3/users/7b6b607fa7bd4f4f8c482424569abe34` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.222199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6b607fa7bd4f4f8c482424569abe34)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.224102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.275306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6b607fa7bd4f4f8c482424569abe34'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.276141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6b607fa7bd4f4f8c482424569abe34'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.288360 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6b607fa7bd4f4f8c482424569abe34'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.297777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6b607fa7bd4f4f8c482424569abe34'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.298484 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-842a6c6e-97df-4c6f-b5fc-b5bbf2bdcc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6b607fa7bd4f4f8c482424569abe34'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.311123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1083/2165] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:37:52 2022] DELETE /identity/v3/users/7b6b607fa7bd4f4f8c482424569abe34 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:52.492651 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.494161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:52.507855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6e80-42ef-4609-90c1-1f79a2ffd232 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.528389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:52.569896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.570546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.570820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.571093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] PATH_INFO: `/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.572276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7ace275bfb469ab65ce79fb22cf0af)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.574199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.624330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7ace275bfb469ab65ce79fb22cf0af', 'request_id': 'req-7e961f5b-190b-4086-a279-c229738757fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.630828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7ace275bfb469ab65ce79fb22cf0af', 'request_id': 'req-7e961f5b-190b-4086-a279-c229738757fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.640310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7ace275bfb469ab65ce79fb22cf0af', 'request_id': 'req-7e961f5b-190b-4086-a279-c229738757fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.651415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] The token cache is being invalidate because project ec7ace275bfb469ab65ce79fb22cf0af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:37:52.651674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.652440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7e961f5b-190b-4086-a279-c229738757fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:52.654127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1083/2166] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:37:52 2022] DELETE /identity/v3/projects/ec7ace275bfb469ab65ce79fb22cf0af => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:52.659280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf500b-4170-4465-9346-86ac1725838f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.659761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf500b-4170-4465-9346-86ac1725838f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.659761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf500b-4170-4465-9346-86ac1725838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.683882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-71cf500b-4170-4465-9346-86ac1725838f None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:52.685111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-71cf500b-4170-4465-9346-86ac1725838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:52.729674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:52.752910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1084/2167] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:52.761211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.761489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.761757 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.782476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:52.783714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:52.792421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1084/2168] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:52.796803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf500b-4170-4465-9346-86ac1725838f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.807044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf500b-4170-4465-9346-86ac1725838f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.814439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.815051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.815328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.815597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.816307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:37:52.816612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.818249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.830538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1085/2169] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:37:52.834999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.844538 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd37c9-d633-4bc7-8823-1fa1d4f91928 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.849697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.850325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.850588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.850856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.851488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.853458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.886526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1085/2170] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:37:52 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:52.892442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:52.902562 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44c5d-adf7-4c86-a02e-4013cda425f9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:52.962607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:52.963223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:52.963500 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:52.963771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:52.964448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:52.966380 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:52.988954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1086/2171] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:37:52 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:37:52.993242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.003614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84aff-f798-43ad-9468-421d729becc3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.008322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.008921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.009194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.009817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.010471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.012061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.021941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1086/2172] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:37:53.025799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.036091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3e48d-8a06-493b-9104-92fb5c255234 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.040931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.041531 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.041818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.042085 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.042757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.044412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37bd3-f7e9-4a24-9588-8edc01559602 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.055047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1087/2173] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:53.059095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.069490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed3cd0-8838-4fa2-bcd0-6844bb61afc2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.076572 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.077705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.078061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.078415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] PATH_INFO: `/v3/projects/ff2f365e8edf47ed852f98c62ba60c8a/users/9ec0f0abc66b4ecc8aa00e59a04c596b/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.098982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2f365e8edf47ed852f98c62ba60c8a, user_id=9ec0f0abc66b4ecc8aa00e59a04c596b, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.101694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.116345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1087/2174] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:53 2022] PUT /identity/v3/projects/ff2f365e8edf47ed852f98c62ba60c8a/users/9ec0f0abc66b4ecc8aa00e59a04c596b/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:53.126853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.127140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.127411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.155627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] MFA Rules not processed for user `9ec0f0abc66b4ecc8aa00e59a04c596b`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:53.157082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:53.220633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1088/2175] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:53.225306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.274646 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:53.290695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122c8f-8939-4e25-9b4a-1ce792eb6b72 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.295534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.296174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.296454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.296719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.297356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.299283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.328022 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1088/2176] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:37:53 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:53.332976 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.346941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3e39-44ee-4681-ae26-7ad0a7e0d147 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.381939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:53.407400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.408021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.408332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.408604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.409249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.411290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.436084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1089/2177] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:37:53 2022] POST /identity/v3/users => generated 307 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:37:53.441574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.452583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf256-7f35-41cf-af4d-905865f98d65 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.458311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.458914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.459186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.459446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.460091 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.461737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.472693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1089/2178] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:37:53.476778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.487239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96756c-4ba6-418f-af23-9b2f846c7108 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.493268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.493951 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.494233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.494506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] PATH_INFO: `/v3/projects/23e380cbedcd41558c4842b5803dd246/users/38e80f6ebc7541328acfe7a6585b1423/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.510655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e380cbedcd41558c4842b5803dd246, user_id=38e80f6ebc7541328acfe7a6585b1423, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.513381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.528582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1090/2179] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:53 2022] PUT /identity/v3/projects/23e380cbedcd41558c4842b5803dd246/users/38e80f6ebc7541328acfe7a6585b1423/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:37:53.532701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.543174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efd8c-e439-4c34-854c-6a5b7c066de7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.602211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.602836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.603113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.603382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.604013 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.605674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb079d2-10b3-4a04-b329-a2b4e1a7b850 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.615517 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1090/2180] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:53.619890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.630435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b8ae-069f-44b5-a22d-c285d9143717 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.635971 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.636655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.636929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.637179 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] PATH_INFO: `/v3/projects/23e380cbedcd41558c4842b5803dd246/users/38e80f6ebc7541328acfe7a6585b1423/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.639218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e380cbedcd41558c4842b5803dd246, user_id=38e80f6ebc7541328acfe7a6585b1423, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.641933 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.652589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1091/2181] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:53 2022] PUT /identity/v3/projects/23e380cbedcd41558c4842b5803dd246/users/38e80f6ebc7541328acfe7a6585b1423/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:53.664136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.664451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.664712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.693722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] MFA Rules not processed for user `38e80f6ebc7541328acfe7a6585b1423`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:53.694949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:53.763893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1091/2182] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:53 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:53.775173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.777097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:53.837945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07df7-907a-4862-9901-c8d259701798 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.842852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.843461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.843742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.844037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:53.853292 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:53.854699 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:53.864318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1092/2183] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:53 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:53.948659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:53.959097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f5e38-92b6-4b1c-ae1e-6ba11827788c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:53.963137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:53.963745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:53.964017 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:53.964317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:54.005655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:54.031244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:54.032675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9b81-792e-4e52-a9b7-6029ed33df7a service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:54.041829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1092/2184] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:53 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:58.173805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:58.243228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d64ecc-8973-49bd-8790-ae5646b6cd98 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:58.248357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:58.249272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:58.249542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:58.249818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] PATH_INFO: `/v3/users/9ec0f0abc66b4ecc8aa00e59a04c596b` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:58.250981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec0f0abc66b4ecc8aa00e59a04c596b)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:58.252912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:58.295396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:58.305985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec0f0abc66b4ecc8aa00e59a04c596b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.306639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec0f0abc66b4ecc8aa00e59a04c596b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.318705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec0f0abc66b4ecc8aa00e59a04c596b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.328362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec0f0abc66b4ecc8aa00e59a04c596b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.329073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec0f0abc66b4ecc8aa00e59a04c596b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.342281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1093/2185] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:37:58 2022] DELETE /identity/v3/users/9ec0f0abc66b4ecc8aa00e59a04c596b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:58.346933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:58.347415 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:58.347521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:58.367501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:58.368744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:58.439097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1093/2186] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:58.449219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:58.450652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:58.508423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0be6-d54b-4f6f-a6c6-c0411e9ed218 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:58.512677 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:58.513286 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:58.513559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:58.513828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:58.523254 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:58.524719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:58.534401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1094/2187] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:37:58.811696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:58.813344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:58.825992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30382a1c-16c8-45de-8df6-31b449abe8c8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:58.831949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:58.832725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:58.833093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:58.833411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] PATH_INFO: `/v3/projects/ff2f365e8edf47ed852f98c62ba60c8a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:58.834695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2f365e8edf47ed852f98c62ba60c8a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:58.836759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:58.884294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2f365e8edf47ed852f98c62ba60c8a', 'request_id': 'req-d09745e7-2472-48a0-ad24-0812e667b8f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.891093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2f365e8edf47ed852f98c62ba60c8a', 'request_id': 'req-d09745e7-2472-48a0-ad24-0812e667b8f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.900630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2f365e8edf47ed852f98c62ba60c8a', 'request_id': 'req-d09745e7-2472-48a0-ad24-0812e667b8f4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.912143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] The token cache is being invalidate because project ff2f365e8edf47ed852f98c62ba60c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:37:58.912439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.913163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:58.914790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1094/2188] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:37:58 2022] DELETE /identity/v3/projects/ff2f365e8edf47ed852f98c62ba60c8a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:58.918983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:58.929119 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bac9b-b90c-4b4d-98bc-349f4e088681 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:58.988563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:58.989194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:58.989468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:58.989738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] PATH_INFO: `/v3/users/38e80f6ebc7541328acfe7a6585b1423` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:58.990892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e80f6ebc7541328acfe7a6585b1423)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:58.992841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.041210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e80f6ebc7541328acfe7a6585b1423'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.041801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e80f6ebc7541328acfe7a6585b1423'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.048861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:59.055393 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e80f6ebc7541328acfe7a6585b1423'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.064896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e80f6ebc7541328acfe7a6585b1423'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.065657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6d5b1e16-2655-4445-97ce-5228de049272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e80f6ebc7541328acfe7a6585b1423'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.078019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1095/2189] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:37:58 2022] DELETE /identity/v3/users/38e80f6ebc7541328acfe7a6585b1423 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:59.354493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.355945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:59.371666 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09745e7-2472-48a0-ad24-0812e667b8f4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.431644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.432350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.432699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.432894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] PATH_INFO: `/v3/projects/23e380cbedcd41558c4842b5803dd246` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.434235 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e380cbedcd41558c4842b5803dd246)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.436249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.442853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:37:59.486940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e380cbedcd41558c4842b5803dd246', 'request_id': 'req-44a9781e-9996-4c3a-9d39-5febb35e8f08'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.493629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e380cbedcd41558c4842b5803dd246', 'request_id': 'req-44a9781e-9996-4c3a-9d39-5febb35e8f08'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.502694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e380cbedcd41558c4842b5803dd246', 'request_id': 'req-44a9781e-9996-4c3a-9d39-5febb35e8f08'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.513439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] The token cache is being invalidate because project 23e380cbedcd41558c4842b5803dd246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:37:59.513721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.514424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-44a9781e-9996-4c3a-9d39-5febb35e8f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:37:59.515948 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1095/2190] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:37:59 2022] DELETE /identity/v3/projects/23e380cbedcd41558c4842b5803dd246 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:59.520993 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.521468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.521567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.542339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:59.543563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:59.606074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1096/2191] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:59.614171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.614451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.614743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.635819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:37:59.637039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:37:59.645595 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1096/2192] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:37:59.649893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.659746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708ecc-a18a-4c22-84a3-feae5c9fcd5a None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.664685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.665293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.665568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.665838 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.666487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:37:59.666785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.668408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.680192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1097/2193] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:37:59.684420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.693871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e54fe-7be6-4b28-b216-0647dacd35cd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.699042 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.699634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.699907 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.700184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.700836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.702736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.739280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1097/2194] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:37:59 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:59.743747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.755867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ac24-2dce-418e-9415-a9286dcc2bcf admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.815175 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.815812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.816089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.816419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.817092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.819020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.842131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1098/2195] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:37:59 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:37:59.846318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.856973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cd1ac-06bd-475e-8cba-dfac4bbe8dcb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.861997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.862609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.862885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.863212 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.863900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.865936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.876077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1098/2196] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:59.880060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.890792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707971-708b-4283-ba91-65684c710bf4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.895713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.896327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.896618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.896881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.897521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.899083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de6262-cb6f-4040-b780-669e4c867045 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.909785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1099/2197] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:37:59.913881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:37:59.925499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009ba8-446e-46b0-b5e3-b90fcc040e58 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:37:59.930914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:37:59.931558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.931878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.932161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] PATH_INFO: `/v3/projects/36fe9a31281d471da7383ea401c50fab/users/38ad2e5cb644431a8f3dac139b545d67/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:37:59.949648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=36fe9a31281d471da7383ea401c50fab, user_id=38ad2e5cb644431a8f3dac139b545d67, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:37:59.952445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:37:59.969145 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1099/2198] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:37:59 2022] PUT /identity/v3/projects/36fe9a31281d471da7383ea401c50fab/users/38ad2e5cb644431a8f3dac139b545d67/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:37:59.980144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:37:59.980826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:37:59.980826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.009997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] MFA Rules not processed for user `38ad2e5cb644431a8f3dac139b545d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:00.011220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:00.042584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:00.081518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1100/2199] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:37:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:38:00.086220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.150442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece1c3e-7c61-4f9e-a3ce-6c45f290f03c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.155605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.156104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.156521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.156875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.157539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.159692 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.187523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1100/2200] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:00 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:00.191711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.202522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553a7ba-1a54-4fa5-b5bd-d2daec7a3754 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.260967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.261578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.261827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.262095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.262739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.264708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.286457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1101/2201] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:00 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:00.290660 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.301940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5ec31-31ef-4a90-b472-2d9343fc40d6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.307246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.307861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.308165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.308443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.309104 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.310758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.321937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1101/2202] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:00.326376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.337457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b431-99d6-4874-8993-518fcb35dc3f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.342910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.343577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.343872 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.344144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] PATH_INFO: `/v3/projects/23aa95113baa4f52806804ccd3250f85/users/a888ac99439549f0b83716848d8375c8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.360090 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23aa95113baa4f52806804ccd3250f85, user_id=a888ac99439549f0b83716848d8375c8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.362773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.377805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1102/2203] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:00 2022] PUT /identity/v3/projects/23aa95113baa4f52806804ccd3250f85/users/a888ac99439549f0b83716848d8375c8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:00.381884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.388941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:00.393180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704df3c6-4b69-4228-9b29-ecdad5ba6022 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.451459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.452077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.452395 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.452669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.453312 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.454920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f970a3-afa3-459a-b8c9-6a53941fa082 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.464735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1102/2204] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:00.470325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.481115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eeb06-ef0d-45ac-ba33-fbf42c1dd57a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.486391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.487073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.487350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.487618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] PATH_INFO: `/v3/projects/23aa95113baa4f52806804ccd3250f85/users/a888ac99439549f0b83716848d8375c8/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.489609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=23aa95113baa4f52806804ccd3250f85, user_id=a888ac99439549f0b83716848d8375c8, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.492250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.502956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1103/2205] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:00 2022] PUT /identity/v3/projects/23aa95113baa4f52806804ccd3250f85/users/a888ac99439549f0b83716848d8375c8/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:00.515352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.515673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.516006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.549952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] MFA Rules not processed for user `a888ac99439549f0b83716848d8375c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:00.551148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:00.621471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1103/2206] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:38:00.632755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:00.635027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:00.694083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f832d-184c-4b0d-8d7b-d785e55730ae admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:00.698592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:00.699196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:00.699465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:00.699729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:00.710266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:00.711672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b058f-0c49-48be-8554-77ae9884f5d1 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:00.722879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1104/2207] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:01.331889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:01.396503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850e08c-bcd0-4239-a4b2-a348d40d5500 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:01.401353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:01.401980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:01.402251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:01.402515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] PATH_INFO: `/v3/users/38ad2e5cb644431a8f3dac139b545d67` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:01.403794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ad2e5cb644431a8f3dac139b545d67)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:01.405784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:01.436491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:01.454721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ad2e5cb644431a8f3dac139b545d67'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:01.461514 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ad2e5cb644431a8f3dac139b545d67'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:01.474607 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ad2e5cb644431a8f3dac139b545d67'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:01.475299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ad2e5cb644431a8f3dac139b545d67'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:01.483551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ad2e5cb644431a8f3dac139b545d67'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:01.489510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1104/2208] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:01 2022] DELETE /identity/v3/users/38ad2e5cb644431a8f3dac139b545d67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:01.494235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:01.494551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:01.494829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:01.514592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:01.515912 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:01.534975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:01.578062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1105/2209] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:38:01.588940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:01.590726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:01.648890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecaca46-eb7c-40c8-ac9d-c4d10c0583c0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:01.653093 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:01.653708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:01.653983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:01.654249 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:01.663535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:01.664990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:01.674327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1105/2210] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:01.950185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:01.951711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:01.965665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea35fda-78f1-43c8-8022-2218bb544fdb None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:01.971856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:01.973369 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:01.973643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:01.974018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] PATH_INFO: `/v3/projects/36fe9a31281d471da7383ea401c50fab` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:01.975212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36fe9a31281d471da7383ea401c50fab)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:01.977075 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:02.028409 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36fe9a31281d471da7383ea401c50fab', 'request_id': 'req-6edc1ed2-1ff1-4764-96cf-ea04ed577899'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.034963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36fe9a31281d471da7383ea401c50fab', 'request_id': 'req-6edc1ed2-1ff1-4764-96cf-ea04ed577899'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.043673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36fe9a31281d471da7383ea401c50fab', 'request_id': 'req-6edc1ed2-1ff1-4764-96cf-ea04ed577899'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.053952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] The token cache is being invalidate because project 36fe9a31281d471da7383ea401c50fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:02.054243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.054882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.056402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1106/2211] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:01 2022] DELETE /identity/v3/projects/36fe9a31281d471da7383ea401c50fab => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:02.060650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:02.070876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540239-8b4f-4033-848b-9f8b1ef809a7 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:02.131041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:02.131691 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.131971 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.132301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] PATH_INFO: `/v3/users/a888ac99439549f0b83716848d8375c8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.133442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a888ac99439549f0b83716848d8375c8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:02.135854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:02.185877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a888ac99439549f0b83716848d8375c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.187977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:02.194519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a888ac99439549f0b83716848d8375c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.206895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a888ac99439549f0b83716848d8375c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.207556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a888ac99439549f0b83716848d8375c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.215888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2736f8a8-8446-41a4-ab5f-570ccf112ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a888ac99439549f0b83716848d8375c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.222110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1106/2212] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:02 2022] DELETE /identity/v3/users/a888ac99439549f0b83716848d8375c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:38:02.512486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:02.514152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:02.528029 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc1ed2-1ff1-4764-96cf-ea04ed577899 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:02.591539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:02.592184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.592469 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.592757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] PATH_INFO: `/v3/projects/23aa95113baa4f52806804ccd3250f85` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.593908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23aa95113baa4f52806804ccd3250f85)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:02.595844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:02.645158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23aa95113baa4f52806804ccd3250f85', 'request_id': 'req-9780533f-d57a-484e-ba75-6cdf71d9c5c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.646244 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:02.653825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23aa95113baa4f52806804ccd3250f85', 'request_id': 'req-9780533f-d57a-484e-ba75-6cdf71d9c5c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.662836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23aa95113baa4f52806804ccd3250f85', 'request_id': 'req-9780533f-d57a-484e-ba75-6cdf71d9c5c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.674411 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] The token cache is being invalidate because project 23aa95113baa4f52806804ccd3250f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:02.674710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.675334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9780533f-d57a-484e-ba75-6cdf71d9c5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:02.676888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1107/2213] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:02 2022] DELETE /identity/v3/projects/23aa95113baa4f52806804ccd3250f85 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:02.681851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.682304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.682407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.702224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:02.703437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:02.771173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1107/2214] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:02.779476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.779759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.780037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.801610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:02.802836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:02.811434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1108/2215] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:38:02.815916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:02.825997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a886-5a01-4df0-bd8d-fa3d7ff4d36a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:02.830845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:02.831462 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.831737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.831998 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.832669 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:02.832970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:02.834579 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:02.847438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1108/2216] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:38:02.851573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:02.862513 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23325463-ded7-47b1-8cd5-41f960e895d9 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:02.871100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:02.871969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.872364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.872731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.873686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:02.876377 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:02.908165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1109/2217] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:02 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:02.912460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:02.922778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd073b8-6fe5-48f0-a26d-100a5e7c5491 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:02.982288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:02.982912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:02.983217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:02.983479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:02.984226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:02.986080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.008867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1109/2218] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:02 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:03.013459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.024027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a4c0b-60c2-48e3-a8b9-5a8a4bbd6787 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.029146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.029750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.030030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.030295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.030947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.032630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.043151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1110/2219] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:03.047060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.058025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a52cf-cde1-42bb-9a88-df1b10873703 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.064002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.064682 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.064962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.065231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.065943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.067684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5b2f2-ad71-47d4-9505-f08c81745581 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.079076 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1110/2220] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:03.083697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.094104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8c96b-affe-4a6d-930f-8e0391f6966a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.099443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.100043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.100370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.100654 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] PATH_INFO: `/v3/projects/538a8be5d0e445aabccae7d03928de08/users/3fef05edd0bb4d1ea3f854a9903329ae/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.119558 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538a8be5d0e445aabccae7d03928de08, user_id=3fef05edd0bb4d1ea3f854a9903329ae, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.122420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.137240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1111/2221] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:03 2022] PUT /identity/v3/projects/538a8be5d0e445aabccae7d03928de08/users/3fef05edd0bb4d1ea3f854a9903329ae/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:03.147780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.148060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.148376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.176675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] MFA Rules not processed for user `3fef05edd0bb4d1ea3f854a9903329ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:03.177902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:03.198137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:03.244734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1111/2222] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:03.249357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.312239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c45d75-c62d-4c8b-8964-b67ea16eb723 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.317682 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.318305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.318580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.318842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.319504 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.321503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.350839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1112/2223] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:03 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:38:03.354912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.365986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a700b5-d4db-42fe-aabe-fe7300b508df None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.422764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.423372 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.423647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.423917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.424602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.426546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.450521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1112/2224] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:03 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:03.456021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.467164 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a07f0-6813-4d27-a6cb-46c1c14ac776 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.473130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.473729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.474001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.474268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.474917 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.476559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.487576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1113/2225] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:03.491790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.502591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006a46-c45f-4d02-806d-67374c79c398 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.508444 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.509107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.509730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.509997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] PATH_INFO: `/v3/projects/c440505b01bf4871b18c0eec93f46c82/users/192237cd6b024bce864a4ae4a642e836/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.525769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c440505b01bf4871b18c0eec93f46c82, user_id=192237cd6b024bce864a4ae4a642e836, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.528464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.542788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1113/2226] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:03 2022] PUT /identity/v3/projects/c440505b01bf4871b18c0eec93f46c82/users/192237cd6b024bce864a4ae4a642e836/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:03.546817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.556926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ab70-dfbf-4861-ac57-7f0a8e7d2db3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.602887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:03.617052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.617674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.617957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.618225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.618871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.620543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c5cb-3de3-41f3-b5da-1adc84db707a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.630389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1114/2227] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:03.635106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.645571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ad0-ab01-4c43-8862-4d982b266e2e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.650270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.650922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.651173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.651459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] PATH_INFO: `/v3/projects/c440505b01bf4871b18c0eec93f46c82/users/192237cd6b024bce864a4ae4a642e836/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.653380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c440505b01bf4871b18c0eec93f46c82, user_id=192237cd6b024bce864a4ae4a642e836, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.656012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.666539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1114/2228] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:03 2022] PUT /identity/v3/projects/c440505b01bf4871b18c0eec93f46c82/users/192237cd6b024bce864a4ae4a642e836/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:03.678027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.678318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.678813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.706832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] MFA Rules not processed for user `192237cd6b024bce864a4ae4a642e836`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:03.708036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:03.784250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1115/2229] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:03.799906 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:03.802178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:03.862854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e177-f6d7-440a-9acd-58317db2ab1c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:03.866727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:03.867326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:03.867600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:03.867862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:03.877134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:03.878574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facfcb9-b6a3-4c36-a244-fa6ccf3a975d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:03.888321 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1115/2230] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:04.281960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:04.351115 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c85f9-335d-438e-8111-5eb9a56228fa None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:04.355888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:04.356561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:04.356859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:04.357123 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] PATH_INFO: `/v3/users/3fef05edd0bb4d1ea3f854a9903329ae` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:04.358324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fef05edd0bb4d1ea3f854a9903329ae)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:04.360352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:04.412467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fef05edd0bb4d1ea3f854a9903329ae'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:04.413023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fef05edd0bb4d1ea3f854a9903329ae'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:04.424667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fef05edd0bb4d1ea3f854a9903329ae'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:04.433695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fef05edd0bb4d1ea3f854a9903329ae'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:04.434420 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fef05edd0bb4d1ea3f854a9903329ae'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:04.435640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:04.448862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1116/2231] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:04 2022] DELETE /identity/v3/users/3fef05edd0bb4d1ea3f854a9903329ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:04.453441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:04.453724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:04.453994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:04.474509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:04.475753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:04.488143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:04.538592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1116/2232] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:04.550230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:04.551909 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:04.613873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612928ae-237a-47fe-b92a-462fd16b3be9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:04.617887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:04.618497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:04.618766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:04.619036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:04.628517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:04.629956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:04.639954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1117/2233] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:04.925163 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:04.926627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:04.938925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33829f4-c954-4876-91bb-2409483d4855 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:04.944683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:04.945383 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:04.945656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:04.945913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] PATH_INFO: `/v3/projects/538a8be5d0e445aabccae7d03928de08` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:04.947132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=538a8be5d0e445aabccae7d03928de08)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:04.950066 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:04.999751 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538a8be5d0e445aabccae7d03928de08', 'request_id': 'req-22bc465a-c11a-4eed-aab7-0f56832d8704'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.006100 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538a8be5d0e445aabccae7d03928de08', 'request_id': 'req-22bc465a-c11a-4eed-aab7-0f56832d8704'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.015292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538a8be5d0e445aabccae7d03928de08', 'request_id': 'req-22bc465a-c11a-4eed-aab7-0f56832d8704'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.026282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] The token cache is being invalidate because project 538a8be5d0e445aabccae7d03928de08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:05.026585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.027308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.028960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1117/2234] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:04 2022] DELETE /identity/v3/projects/538a8be5d0e445aabccae7d03928de08 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:05.033450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.043346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50325aac-acea-4420-a118-df40935cf367 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:05.106627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:05.107270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.107550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.107824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] PATH_INFO: `/v3/users/192237cd6b024bce864a4ae4a642e836` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.109121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192237cd6b024bce864a4ae4a642e836)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:05.111151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:05.160721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192237cd6b024bce864a4ae4a642e836'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.161410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192237cd6b024bce864a4ae4a642e836'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.173490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192237cd6b024bce864a4ae4a642e836'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.181689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192237cd6b024bce864a4ae4a642e836'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.182315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4995c814-a1e9-4b25-8680-2755cba78bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192237cd6b024bce864a4ae4a642e836'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.194501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1118/2235] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:05 2022] DELETE /identity/v3/users/192237cd6b024bce864a4ae4a642e836 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:05.480001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.481635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:05.495745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc465a-c11a-4eed-aab7-0f56832d8704 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:05.555359 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:05.555958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.556262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.556528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] PATH_INFO: `/v3/projects/c440505b01bf4871b18c0eec93f46c82` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.557674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c440505b01bf4871b18c0eec93f46c82)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:05.559583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:05.603271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:05.607443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c440505b01bf4871b18c0eec93f46c82', 'request_id': 'req-78d579e5-0454-4170-86b5-7e45fb135ffe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.613925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c440505b01bf4871b18c0eec93f46c82', 'request_id': 'req-78d579e5-0454-4170-86b5-7e45fb135ffe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.623008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c440505b01bf4871b18c0eec93f46c82', 'request_id': 'req-78d579e5-0454-4170-86b5-7e45fb135ffe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.633231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] The token cache is being invalidate because project c440505b01bf4871b18c0eec93f46c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:05.633512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.634113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-78d579e5-0454-4170-86b5-7e45fb135ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:05.636001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1118/2236] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:05 2022] DELETE /identity/v3/projects/c440505b01bf4871b18c0eec93f46c82 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:05.640925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.641410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.641508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.661832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:05.663751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:05.673068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:05.737021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1119/2237] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:05.745442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.745728 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.745994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.768343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:05.769581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:05.779329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1119/2238] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:05.784405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.799475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a82a1-ce92-4723-b3d9-c87aa9086f22 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:05.808427 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:05.809159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.809456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.809771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.810722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:05.811147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:05.813624 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:05.826084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1120/2239] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:38:05.830813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.840713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34067a-1074-4152-a637-ac8ed1a31aac None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:05.845927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:05.846510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.846782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.847049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.847707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:05.849642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:05.882330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1120/2240] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:05 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:05.886956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.897297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6a51-6f7c-481e-a386-14e13aa63f97 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:05.955096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:05.955711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:05.955989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:05.956291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:05.956935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:05.958886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:05.983643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1121/2241] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:05 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:05.988171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:05.999012 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37905f-5453-41d2-b378-6034a8180770 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:06.004638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:06.005274 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:06.005566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:06.005834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:06.006504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:06.008197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:06.019159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1121/2242] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:06.023069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:06.033568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5e334-c1b9-4751-bb89-4038188a0309 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:06.039302 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:06.039900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:06.040233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:06.040496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:06.041145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:06.042745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ac669-b23c-4b74-8b04-81bd1492ae54 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:06.054126 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1122/2243] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:06.058332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:06.069347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747774-bf2f-4044-ad2b-34e1fee79963 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:06.075021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:06.075679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:06.075957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:06.076273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] PATH_INFO: `/v3/projects/ef72a281380a4b3e993991b33d59604e/users/cb07fa67b64645a58e37fa9b607fa83f/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:06.092350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef72a281380a4b3e993991b33d59604e, user_id=cb07fa67b64645a58e37fa9b607fa83f, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:06.095020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:06.110909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1122/2244] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:06 2022] PUT /identity/v3/projects/ef72a281380a4b3e993991b33d59604e/users/cb07fa67b64645a58e37fa9b607fa83f/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:06.122570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:06.122847 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:06.123117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:06.157805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] MFA Rules not processed for user `cb07fa67b64645a58e37fa9b607fa83f`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:06.159678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:06.231636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1123/2245] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:06.244150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:06.246493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:06.306603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6bd61-b8cb-48cc-b2e2-0625f70b972a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:06.310479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:06.311086 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:06.311360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:06.311623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:06.320231 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:06.321647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:06.330617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1123/2246] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:16.228833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:16.326623 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:20.272402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:20.274136 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:20.332424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc44b-337d-4b59-a441-3f8f8da60a3f None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:20.336445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:20.337053 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:20.337325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:20.337598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:20.339578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:20.349443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:20.350850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:20.358902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1124/2247] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:29.186694 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:29.188901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:29.310721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86779-2cbc-4b21-ba7d-e7c6ecb871e4 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:29.321632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:29.322308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:29.322580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:29.322851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:29.324978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:29.351299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:29.353114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84944f0-0008-411f-a68e-9c3ab06b1eaa service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:29.354101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:29.364833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1124/2248] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:29.470727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999949d-7e6b-40a6-9223-70c931d12d78 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:29.476169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:29.476334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:29.476708 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:29.476985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] PATH_INFO: `/v3/users/cb07fa67b64645a58e37fa9b607fa83f` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:29.478616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb07fa67b64645a58e37fa9b607fa83f)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:29.480487 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:29.532588 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb07fa67b64645a58e37fa9b607fa83f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:29.533231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb07fa67b64645a58e37fa9b607fa83f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:29.545190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb07fa67b64645a58e37fa9b607fa83f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:29.554550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb07fa67b64645a58e37fa9b607fa83f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:29.555197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb07fa67b64645a58e37fa9b607fa83f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:29.569042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1125/2249] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:29 2022] DELETE /identity/v3/users/cb07fa67b64645a58e37fa9b607fa83f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:38:29.574343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:29.574624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:29.574895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:29.595277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:29.596531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:29.660003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1125/2250] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:29.669007 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:29.670465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:29.697554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:29.732991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fb17-6b74-4296-9b23-f9e21dcd98cb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:29.737052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:29.737742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:29.738088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:29.738422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:29.750883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:29.751109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d660d-8059-491f-b06c-1e2b57b7fd2b service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:29.763462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1126/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:30.037599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.039078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:30.050990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0cebe-9f40-4534-925b-f42e4ed43ef1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.055598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.056289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.056569 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.056861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] PATH_INFO: `/v3/projects/ef72a281380a4b3e993991b33d59604e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.057990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef72a281380a4b3e993991b33d59604e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.059945 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.110192 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef72a281380a4b3e993991b33d59604e', 'request_id': 'req-49284a42-738f-432a-8459-40fbf5f3e5c4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:30.117124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef72a281380a4b3e993991b33d59604e', 'request_id': 'req-49284a42-738f-432a-8459-40fbf5f3e5c4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:30.127188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef72a281380a4b3e993991b33d59604e', 'request_id': 'req-49284a42-738f-432a-8459-40fbf5f3e5c4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:30.141854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] The token cache is being invalidate because project ef72a281380a4b3e993991b33d59604e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:30.142237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:30.142957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49284a42-738f-432a-8459-40fbf5f3e5c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:30.144952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1126/2252] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:30 2022] DELETE /identity/v3/projects/ef72a281380a4b3e993991b33d59604e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:30.149828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.150187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.150962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.179104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:30.180614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:30.255354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1127/2253] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:30.263719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.264772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.265223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.288102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:30.289570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:30.300986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1127/2254] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:30.305375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.317680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a4c4a-8699-446b-bd15-f55525ce3c57 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.322607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.323511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.323791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.324190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.324787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:30.325092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.326771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.342653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1128/2255] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:38:30.344166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.348899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:30.355215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c5bd5-09db-48d2-acfd-0f5e0bab2cd0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.365005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.365926 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.366914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.367561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.368648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.371668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.415188 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1128/2256] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:30 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:38:30.419362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.429640 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecff393-ca02-4a66-8230-3dab9f8df9bf admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.490563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.491189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.491467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.491773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.492489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.494458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.521023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1129/2257] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:30 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:38:30.522386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.532772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0848-03f5-470f-8d6c-d128dc3cd34d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.538105 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.538657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.538929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.539254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.539908 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.541685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.554631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1129/2258] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:30.555829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.566710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ae06a-51b7-4637-8b51-9df36d53eec4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.577736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.578370 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.578655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.578922 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.579583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.581227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192c47-5fb8-44cc-bf80-c9fbf1d66af2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.593895 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1130/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:38:30.597903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.608886 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad0ebb-e6fd-40b5-94e8-aeaad87dae5e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.614134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.614798 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.615051 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.615350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] PATH_INFO: `/v3/projects/b7aba80eb08842eaa097890e313a6940/users/963b6283caf14c6fb80d96dae95539b0/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.630259 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7aba80eb08842eaa097890e313a6940, user_id=963b6283caf14c6fb80d96dae95539b0, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.633097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.646896 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1130/2260] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:30 2022] PUT /identity/v3/projects/b7aba80eb08842eaa097890e313a6940/users/963b6283caf14c6fb80d96dae95539b0/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:30.658533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.658814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.659079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.688108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] MFA Rules not processed for user `963b6283caf14c6fb80d96dae95539b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:30.689367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:30.712897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:30.764750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1131/2261] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:30.777074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:30.780007 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:30.837756 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f574-8707-4cb7-950d-926233fb100b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:30.844507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:30.845164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:30.845570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:30.845949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:30.857399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:30.859117 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:30.868955 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1131/2262] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:33.318399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:33.319962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:33.377834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398280e-2574-46e8-a709-f7cf613a1bb0 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:33.381807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:33.382417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:33.382690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:33.382955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:33.385265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:33.449953 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:33.451431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c766-10a6-472d-9aba-c40d9fd2f30d service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:33.453360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:33.462016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1132/2263] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:33.464562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6fd4a-5b3e-4ea2-b7b7-22fd87d20071 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:33.470968 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:33.471617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:33.471891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:33.472182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] PATH_INFO: `/v3/users/963b6283caf14c6fb80d96dae95539b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:33.473464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=963b6283caf14c6fb80d96dae95539b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:33.475418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:33.526305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963b6283caf14c6fb80d96dae95539b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:33.537531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963b6283caf14c6fb80d96dae95539b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:33.551771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963b6283caf14c6fb80d96dae95539b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:33.552624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963b6283caf14c6fb80d96dae95539b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:33.560743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963b6283caf14c6fb80d96dae95539b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:33.566819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1132/2264] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:33 2022] DELETE /identity/v3/users/963b6283caf14c6fb80d96dae95539b0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:33.573832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:33.574117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:33.574391 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:33.595117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:33.596390 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:33.660735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1133/2265] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:33.670704 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:33.672263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:33.695642 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:33.737504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401f86c-5008-4cc1-9014-536e0ee511fe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:33.741458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:33.742127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:33.742409 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:33.742680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:33.753151 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:33.754604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bff50-c0fe-4ca1-8f35-2b9af8467777 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:33.765711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1133/2266] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:34.041512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.043072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:34.060825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ce55-ab59-448a-b6ee-59df84d1fc5b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.066795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.067493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.067771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.068176 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] PATH_INFO: `/v3/projects/b7aba80eb08842eaa097890e313a6940` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.069297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7aba80eb08842eaa097890e313a6940)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.071246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.078422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:34.122385 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7aba80eb08842eaa097890e313a6940', 'request_id': 'req-9cf2d1a2-a253-40a8-b833-a17622952eea'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:34.129398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7aba80eb08842eaa097890e313a6940', 'request_id': 'req-9cf2d1a2-a253-40a8-b833-a17622952eea'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:34.137982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7aba80eb08842eaa097890e313a6940', 'request_id': 'req-9cf2d1a2-a253-40a8-b833-a17622952eea'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:34.149229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] The token cache is being invalidate because project b7aba80eb08842eaa097890e313a6940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:34.149553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:34.150252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9cf2d1a2-a253-40a8-b833-a17622952eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:34.151887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1134/2267] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:34 2022] DELETE /identity/v3/projects/b7aba80eb08842eaa097890e313a6940 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:34.156983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.157511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.157862 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.178690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:34.179891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:34.243421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1134/2268] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:34.311018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.311288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.311582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.331972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:34.333277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:34.341881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1135/2269] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:34.346459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.356814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8687356-05ea-4dd7-83c8-25b70aab5141 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.361743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.362345 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.362617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.362877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.363574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:34.363876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.365530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.377402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1135/2270] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:38:34.381901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.391831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ad68-a56a-40f0-b52a-fc14aafff716 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.397403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.398026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.398299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.398564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.399230 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.401282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.432034 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1136/2271] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:34 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:34.433180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.443744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e40f3-3da7-47e9-bcb5-1ecd3deefd39 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.500606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.501317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.501529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.501744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.502422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.504488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.529338 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1136/2272] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:34 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:34.530683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.542151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd721-eff3-4fdb-a9cc-987b1870ad72 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.547507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.548166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.548448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.548713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.549374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.551584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.566796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1137/2273] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:34.572471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.582738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e900-3b1b-4d34-96a7-a6c511cd39b4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.593389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.594043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.594320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.594774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.595441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.597153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd346c-cc7a-49a3-96fd-0acdcba8ec4b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.612181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1137/2274] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:34.614181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.626346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d803-5e79-4c5b-bf89-869968e24971 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.631569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.633208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.633208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.633208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] PATH_INFO: `/v3/projects/340df5a9e41940d7b010168feb108bd3/users/76d8100e29a24533a014b8d073e45ca5/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.651323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] RBAC: Authorizing `identity:create_grant(project_id=340df5a9e41940d7b010168feb108bd3, user_id=76d8100e29a24533a014b8d073e45ca5, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.654296 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.669816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1138/2275] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:34 2022] PUT /identity/v3/projects/340df5a9e41940d7b010168feb108bd3/users/76d8100e29a24533a014b8d073e45ca5/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:34.676463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.677164 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.677668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.699374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:34.700659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:34.727673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:34.770199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1138/2276] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:34.780783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:34.782512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:34.842248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b00e-0855-4659-9cda-46d038ee66da admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:34.846333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:34.846950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:34.847222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:34.847482 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:34.857272 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:34.858691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:34.868056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1139/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:37.032812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:37.034299 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:37.051647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555d093-f1a8-491e-a3a0-9571003a06ba None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:37.057148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:37.058181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.058656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.059318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] PATH_INFO: `/v3/users/d439d54a4f2348d78ddb84691f2a4f63` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.061698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d439d54a4f2348d78ddb84691f2a4f63)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:37.064037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:37.121199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd439d54a4f2348d78ddb84691f2a4f63'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.128562 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd439d54a4f2348d78ddb84691f2a4f63'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.151228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd439d54a4f2348d78ddb84691f2a4f63'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.152046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd439d54a4f2348d78ddb84691f2a4f63'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.170773 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76e50756-44a5-4fed-8570-ae946325073c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd439d54a4f2348d78ddb84691f2a4f63'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.182469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1139/2278] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:37 2022] DELETE /identity/v3/users/d439d54a4f2348d78ddb84691f2a4f63 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:37.382573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:37.394443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6d05-892a-4f91-85f9-578ce3d71b5b service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:37.479322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:37.537052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:37.537819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.538306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.538762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] PATH_INFO: `/v3/projects/33afb7274a9146cdbee0eeb2e6f48785` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.540212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33afb7274a9146cdbee0eeb2e6f48785)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:37.542375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:37.620665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33afb7274a9146cdbee0eeb2e6f48785', 'request_id': 'req-4f1c47db-9730-4c33-979e-cb287c03ea7d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.628994 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33afb7274a9146cdbee0eeb2e6f48785', 'request_id': 'req-4f1c47db-9730-4c33-979e-cb287c03ea7d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.640822 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33afb7274a9146cdbee0eeb2e6f48785', 'request_id': 'req-4f1c47db-9730-4c33-979e-cb287c03ea7d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.654593 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] The token cache is being invalidate because project 33afb7274a9146cdbee0eeb2e6f48785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:37.654885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.656332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-4f1c47db-9730-4c33-979e-cb287c03ea7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:37.657920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1140/2279] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:37 2022] DELETE /identity/v3/projects/33afb7274a9146cdbee0eeb2e6f48785 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:37.663194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.663480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.663750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.690598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:37.691835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:37.785058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1140/2280] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:37.793906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.794497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.794497 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.820906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:37.822367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:37.835980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1141/2281] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:37.837323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:37.847495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38080b1-de9a-46b9-96c6-6d400e4771fb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:37.856827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:37.858143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.858474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.858938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.860423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:37.860831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:37.862960 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:37.879789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1141/2282] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:38:37.881043 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:37.891856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8221b-1a47-4056-9124-32a8d024ac7d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:37.897014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:37.897727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:37.898074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:37.898413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:37.901882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:37.903986 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:37.944766 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1142/2283] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:37 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:37.946341 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:37.957487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b750a-b19b-45e5-b5ef-74275bc582ef admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:37.989814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:38.026754 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:38.027205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.027489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.027762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.028458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:38.030438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:38.058705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1142/2284] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:37 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:38.062868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:38.074266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916fb39-8099-4468-8980-5bc8030eb97f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:38.080633 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:38.081280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.081566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.081997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.082684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:38.084352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:38.095852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1143/2285] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:38.100058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:38.111069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8896b5-359a-44ef-8fcb-2397dde2c5d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:38.115983 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:38.116664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.116947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.117224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.117891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:38.119620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d133cc-7cf9-43b4-9efd-4a27166011c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:38.136910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:38.137995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1143/2286] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:38.149793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911818-1a1f-4500-9754-afb52d1c8b1f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:38.160049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:38.161510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.161510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.161510 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] PATH_INFO: `/v3/projects/4d5fd6aafa0a486b812d64ae0f6047d5/users/b5f99b3f93c74cda827a6798d09e5118/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.181361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5fd6aafa0a486b812d64ae0f6047d5, user_id=b5f99b3f93c74cda827a6798d09e5118, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:38.184267 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:38.209574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1144/2287] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:38 2022] PUT /identity/v3/projects/4d5fd6aafa0a486b812d64ae0f6047d5/users/b5f99b3f93c74cda827a6798d09e5118/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:38:38.221618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ed6a-7dac-495b-9947-caf6645c8b52 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.222098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ed6a-7dac-495b-9947-caf6645c8b52 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.222268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ed6a-7dac-495b-9947-caf6645c8b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.267130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3291ed6a-7dac-495b-9947-caf6645c8b52 None None] MFA Rules not processed for user `b5f99b3f93c74cda827a6798d09e5118`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:38.268448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3291ed6a-7dac-495b-9947-caf6645c8b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:38.348668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1144/2288] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:38.360470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:38.362515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:38.431162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:38.454387 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fbc4b-3a0a-4460-852e-9405d0398770 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:38.466325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:38.467276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:38.467690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:38.468102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:38.482042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:38.483770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:38.494134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1145/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:39.523888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:39.524490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:39.524490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:39.555360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] MFA Rules not processed for user `76d8100e29a24533a014b8d073e45ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:39.556780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:39.629707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1145/2290] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:39.634711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:39.636506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:39.727141 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ff1f-80b2-4e3d-99e4-89cfbcb9e49a service nova] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:39.731801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:39.732455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:39.732728 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:39.732991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:39.733627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:39.735522 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:39.767580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1146/2291] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:39 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:39.770506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:39.783351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafffb76-c87e-4fe9-9155-9bdb3a5cd89c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:39.808708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:39.843668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:39.844352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:39.844631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:39.844901 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:39.845555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:39.847491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:39.871122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1146/2292] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:39 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:39.875134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:39.887092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c980d-00cf-43ac-a5c3-28f0f01129fb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:39.892158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:39.892796 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:39.893121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:39.893399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:39.894054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:39.895740 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:39.907643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1147/2293] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:39.911887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:39.922624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be29df-42d4-47e5-b1c4-8a24fa6fe4a2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:39.927306 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:39.927982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:39.928336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:39.928605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] PATH_INFO: `/v3/projects/887cdd2afaaa48fe930f5fd87388fc18/users/369f348c7ca44e6baa56bc3e9f7adbb4/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:39.944830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887cdd2afaaa48fe930f5fd87388fc18, user_id=369f348c7ca44e6baa56bc3e9f7adbb4, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:39.947582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:39.962389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1147/2294] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:39 2022] PUT /identity/v3/projects/887cdd2afaaa48fe930f5fd87388fc18/users/369f348c7ca44e6baa56bc3e9f7adbb4/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:39.966548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:39.986883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8b17-56e2-4735-abde-f4bd9c7178d3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:40.051475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:40.052152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:40.052674 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:40.052946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:40.053599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:40.055206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4225c-9757-46b4-964f-157272eda42b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:40.065623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1148/2295] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:40.069947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:40.080676 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffeb7-338b-459c-852c-847463a00bf1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:40.085419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:40.086004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:40.086277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:40.086543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] PATH_INFO: `/v3/projects/887cdd2afaaa48fe930f5fd87388fc18/users/369f348c7ca44e6baa56bc3e9f7adbb4/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:40.088550 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=887cdd2afaaa48fe930f5fd87388fc18, user_id=369f348c7ca44e6baa56bc3e9f7adbb4, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:40.091172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:40.102615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1148/2296] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:40 2022] PUT /identity/v3/projects/887cdd2afaaa48fe930f5fd87388fc18/users/369f348c7ca44e6baa56bc3e9f7adbb4/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:44.457610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:44.458121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:44.458121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:44.486135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] MFA Rules not processed for user `369f348c7ca44e6baa56bc3e9f7adbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:44.487528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:44.535546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:44.562394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1149/2297] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:44.578677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:44.683855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b82f-f34f-40d2-929e-e55ac047bdfc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:44.689701 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:44.690450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:44.690780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:44.691141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:44.763451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:44.765111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6a70c-86b3-43e6-bd55-8749e618654b service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:44.774358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1149/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:47.091744 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:47.169404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269b9bf-3999-4cc6-a6cf-f2b419e59225 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:47.175218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:47.176023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:47.176559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:47.177048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] PATH_INFO: `/v3/users/b5f99b3f93c74cda827a6798d09e5118` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:47.178459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f99b3f93c74cda827a6798d09e5118)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:47.180774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:47.255776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f99b3f93c74cda827a6798d09e5118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.256666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f99b3f93c74cda827a6798d09e5118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.292963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f99b3f93c74cda827a6798d09e5118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.315451 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f99b3f93c74cda827a6798d09e5118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.317704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f99b3f93c74cda827a6798d09e5118'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.346423 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1150/2299] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:47 2022] DELETE /identity/v3/users/b5f99b3f93c74cda827a6798d09e5118 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:47.351068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:47.351795 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:47.351893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:47.376833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:47.378350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:47.393540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:47.463159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1150/2300] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:47.474507 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:47.476352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:47.551446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669437f-d7dc-4c92-abbd-3d48ba78232e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:47.555414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:47.556248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:47.556802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:47.557278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:47.568622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:47.570684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a953ca1-506b-4c83-88b2-e5696f53efe1 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:47.594721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1151/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:47.904949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:47.906503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:47.920727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71b069-1415-43fc-87cc-6fc6f684532a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:47.926090 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:47.926783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:47.927062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:47.927329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] PATH_INFO: `/v3/projects/4d5fd6aafa0a486b812d64ae0f6047d5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:47.928636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5fd6aafa0a486b812d64ae0f6047d5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:47.930715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:47.987857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5fd6aafa0a486b812d64ae0f6047d5', 'request_id': 'req-0c667db1-4025-4930-a865-423da2a4063f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:47.994855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5fd6aafa0a486b812d64ae0f6047d5', 'request_id': 'req-0c667db1-4025-4930-a865-423da2a4063f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:48.004847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5fd6aafa0a486b812d64ae0f6047d5', 'request_id': 'req-0c667db1-4025-4930-a865-423da2a4063f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:48.017531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] The token cache is being invalidate because project 4d5fd6aafa0a486b812d64ae0f6047d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:48.017818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:48.018475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c667db1-4025-4930-a865-423da2a4063f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:48.020069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1151/2302] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:47 2022] DELETE /identity/v3/projects/4d5fd6aafa0a486b812d64ae0f6047d5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:48.026145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.026327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.026642 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.038259 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:48.053187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:48.054405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:48.117177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1152/2303] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:48.125582 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.125927 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.126302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.151305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:48.152610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:48.160601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1152/2304] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:48.164982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.174915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c0e2c-8be5-44b1-8705-75791f3489f2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.179442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.180101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.180559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.180924 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.181638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:48.181948 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.183658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.197483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1153/2305] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:38:48.202644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.212388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ca70-6621-4e97-a1a3-f66aec39d177 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.222662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.223316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.223587 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.223851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.224528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.226483 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.257618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1153/2306] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:48.262363 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.273009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45438f8-c9d6-49d7-88ca-ac3bb1c51960 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.341819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.342431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.342701 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.342954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.343612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.345573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.370759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1154/2307] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:48.375044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.386707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01651ed-3f74-4d50-8fdd-859258afc0b2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.392069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.392771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.393060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.393333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.394000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.395840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.406537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1154/2308] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:48.410995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.422027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc77456-d613-4b87-93dc-e6bcbeedffe8 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.427479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.428135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.428445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.428710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.429395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.431127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d112fd-6b36-46df-b994-781d9a3b52b8 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.444753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1155/2309] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:48.448969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.461699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa5c76-cc9c-4c1a-a50c-4065524935b2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.466715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.467385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.467663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.467939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] PATH_INFO: `/v3/projects/32ef51cb4d9444dd88e811b2ff0dfb52/users/53cd7e5cde214c2d8791e41ecf8fb62f/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.485375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ef51cb4d9444dd88e811b2ff0dfb52, user_id=53cd7e5cde214c2d8791e41ecf8fb62f, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.488039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.503016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1155/2310] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:48 2022] PUT /identity/v3/projects/32ef51cb4d9444dd88e811b2ff0dfb52/users/53cd7e5cde214c2d8791e41ecf8fb62f/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:48.515609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.515926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.516225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.545989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] MFA Rules not processed for user `53cd7e5cde214c2d8791e41ecf8fb62f`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:48.547206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:48.611680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:48.617768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1156/2311] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:48.622601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.664637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:48.689504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f87098-7690-4db1-bc5b-d16111a55ab3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.694375 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.695004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.695281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.695551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.696220 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.698213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.728179 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1156/2312] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:48.732595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.743405 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458ac02-142d-4e83-8c41-f31321df52bd None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.805539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.806232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.806527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.806811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.807474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.809464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.834177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1157/2313] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:48 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:48.835397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.845999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187de9-207b-4d68-b743-91298da68067 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.850940 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.851535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.851810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.852195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.852835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.854779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.866880 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1157/2314] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:48.871495 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.881793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427be95-36ff-443e-a8b2-ec0ede25651f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:48.888496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:48.889480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:48.889852 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:48.890222 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] PATH_INFO: `/v3/projects/365de72de62047949cfda7819304a44d/users/da506b6747d445e4be8a8ad18a448ed6/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:48.908906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365de72de62047949cfda7819304a44d, user_id=da506b6747d445e4be8a8ad18a448ed6, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:48.911585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:48.927243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1158/2315] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:48 2022] PUT /identity/v3/projects/365de72de62047949cfda7819304a44d/users/da506b6747d445e4be8a8ad18a448ed6/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:38:48.931712 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:48.942047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdffcb7-13c7-4ed7-bad8-4bb73be8befe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:49.002036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:49.002657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:49.003070 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:49.003251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:49.003821 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:49.005488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a124b-d747-465c-901e-32329e1bd561 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:49.015558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1158/2316] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:49.021675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:49.034606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9262b-f5c9-4acf-b6d8-b5d2ffe50c04 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:49.041189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:49.041881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:49.042149 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:49.042416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] PATH_INFO: `/v3/projects/365de72de62047949cfda7819304a44d/users/da506b6747d445e4be8a8ad18a448ed6/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:49.044481 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365de72de62047949cfda7819304a44d, user_id=da506b6747d445e4be8a8ad18a448ed6, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:49.047276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:49.058663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1159/2317] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:49 2022] PUT /identity/v3/projects/365de72de62047949cfda7819304a44d/users/da506b6747d445e4be8a8ad18a448ed6/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:49.069292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:49.069577 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:49.069848 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:49.100909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] MFA Rules not processed for user `da506b6747d445e4be8a8ad18a448ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:49.102191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:49.181260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1159/2318] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:49.197025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:49.198934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:49.264506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb02c-d9b2-47d2-aa4a-089e69243a76 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:49.268900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:49.269452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:49.269726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:49.269991 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:49.279835 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:49.281277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:49.295152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1160/2319] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:49.377035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:49.387378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ac173-ff9e-4c27-af4d-577b94c4f82a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:49.391019 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:49.391912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:49.392255 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:49.392512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:49.427790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:49.464602 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:49.466101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9988f-c16e-4c9d-96a9-b972a3640029 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:49.476058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1160/2320] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:52.297854 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:52.360530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:52.372050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0a29c-a3f7-43bb-87e9-e57ff5cf2e19 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:52.377418 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:52.378069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:52.378345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:52.378612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] PATH_INFO: `/v3/users/53cd7e5cde214c2d8791e41ecf8fb62f` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:52.379846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53cd7e5cde214c2d8791e41ecf8fb62f)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:52.381804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:52.436196 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53cd7e5cde214c2d8791e41ecf8fb62f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:52.436880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53cd7e5cde214c2d8791e41ecf8fb62f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:52.449036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53cd7e5cde214c2d8791e41ecf8fb62f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:52.457311 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53cd7e5cde214c2d8791e41ecf8fb62f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:52.458079 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53cd7e5cde214c2d8791e41ecf8fb62f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:52.471386 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1161/2321] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:52 2022] DELETE /identity/v3/users/53cd7e5cde214c2d8791e41ecf8fb62f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:52.476518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:52.477162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:52.477162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:52.497332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:52.498708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:52.563520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1161/2322] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:52.573702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:52.575247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:52.638323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec0d3f-6caf-4f18-8d1e-39af8b164838 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:52.642875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:52.643538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:52.643837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:52.644103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:52.654690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:52.656716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:52.672392 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1162/2323] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:52.950775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:52.952297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:52.965122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed12c5e-9a8b-40fd-9eb0-cde916ee0065 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:52.970542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:52.971134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:52.971536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:52.971825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] PATH_INFO: `/v3/projects/32ef51cb4d9444dd88e811b2ff0dfb52` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:52.973058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ef51cb4d9444dd88e811b2ff0dfb52)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:52.975028 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:53.026442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ef51cb4d9444dd88e811b2ff0dfb52', 'request_id': 'req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.032976 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ef51cb4d9444dd88e811b2ff0dfb52', 'request_id': 'req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.041762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ef51cb4d9444dd88e811b2ff0dfb52', 'request_id': 'req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.052570 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] The token cache is being invalidate because project 32ef51cb4d9444dd88e811b2ff0dfb52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:53.052855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.053506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.055045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1162/2324] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:52 2022] DELETE /identity/v3/projects/32ef51cb4d9444dd88e811b2ff0dfb52 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:53.059703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:53.070548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ca03-0d50-4912-b6d3-e254f25cca73 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:53.139147 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:53.139901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.140270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.140645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] PATH_INFO: `/v3/users/da506b6747d445e4be8a8ad18a448ed6` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.142235 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] RBAC: Authorizing `identity:delete_user(user_id=da506b6747d445e4be8a8ad18a448ed6)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:53.149731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:53.167440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:53.202130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da506b6747d445e4be8a8ad18a448ed6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.204572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da506b6747d445e4be8a8ad18a448ed6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.217879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da506b6747d445e4be8a8ad18a448ed6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.226891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da506b6747d445e4be8a8ad18a448ed6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.227444 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-bded6f2a-6f72-4efa-9131-fe2a81e9204b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da506b6747d445e4be8a8ad18a448ed6'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.240972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1163/2325] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:38:53 2022] DELETE /identity/v3/users/da506b6747d445e4be8a8ad18a448ed6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:53.524092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:53.525663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:53.541063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa9a2-87ea-44a6-84c4-1f76d132b40d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:53.597398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:53.607921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:53.608891 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.609311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.609775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] PATH_INFO: `/v3/projects/365de72de62047949cfda7819304a44d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.611261 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365de72de62047949cfda7819304a44d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:53.613308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:53.665505 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365de72de62047949cfda7819304a44d', 'request_id': 'req-0187b4b7-7062-40e0-ab51-3f563750c193'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.672800 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365de72de62047949cfda7819304a44d', 'request_id': 'req-0187b4b7-7062-40e0-ab51-3f563750c193'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.683258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365de72de62047949cfda7819304a44d', 'request_id': 'req-0187b4b7-7062-40e0-ab51-3f563750c193'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.694851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] The token cache is being invalidate because project 365de72de62047949cfda7819304a44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:38:53.695204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.695929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0187b4b7-7062-40e0-ab51-3f563750c193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:38:53.697637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1163/2326] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:38:53 2022] DELETE /identity/v3/projects/365de72de62047949cfda7819304a44d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:53.703982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.704833 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.705113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.725867 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:53.727217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:53.792942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1164/2327] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:53.801111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.801461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.801796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.822730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:53.823958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:53.832748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1164/2328] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:53.837065 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:53.847440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f169c0-2585-4ce7-b202-89aa42fc957b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:53.852457 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:53.853076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.853352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.853619 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.854274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:38:53.854579 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:53.856433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:53.869091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1165/2329] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:38:53.873841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:53.883405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8f8d0-ccdf-4905-bfac-8578c0ade4ea None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:53.888941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:53.889583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:53.889830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:53.890095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:53.890776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:53.892779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:53.925860 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1165/2330] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:53 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:53.930493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:53.943291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777b16-adf8-4a24-bb08-340a75d687a2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.004973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.005653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.005937 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.006210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.006959 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.009328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.036702 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1166/2331] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:53 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:54.040950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.052284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb330f25-3cc4-4b1e-b89b-8c8641a77459 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.058243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.058883 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.059190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.059439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.060143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.061778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.074913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1166/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:54.079830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.091069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257f25a-9abc-4757-893b-bec48f4b1290 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.096159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.096767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.097038 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.097303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.097946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.099530 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65035c27-70cb-41e7-a94e-b672c4b31c3b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.104785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:54.111733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1167/2333] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:54.116094 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.126392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5fbb3-d20f-4166-9dbd-9d791742a5e6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.131205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.131878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.132142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.132440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] PATH_INFO: `/v3/projects/6435d0cb59c34bd48e559477b1a02ea0/users/b59655017cc547c5b47b698b9a0cbf21/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.148640 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6435d0cb59c34bd48e559477b1a02ea0, user_id=b59655017cc547c5b47b698b9a0cbf21, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.151551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.167311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1167/2334] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:54 2022] PUT /identity/v3/projects/6435d0cb59c34bd48e559477b1a02ea0/users/b59655017cc547c5b47b698b9a0cbf21/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:54.179383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.179658 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.179923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.210764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] MFA Rules not processed for user `b59655017cc547c5b47b698b9a0cbf21`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:54.211982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:54.287888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1168/2335] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:54.292744 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.362563 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d82e-46b3-44f1-8b03-2f758809839c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.367379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.368006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.368311 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.368583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.369223 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.371181 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.400957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1168/2336] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:54.405280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.417019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec4522-5fac-4fe1-b3fb-50ab6a8283a7 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.479124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.480059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.480374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.480662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.481505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.483606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.507521 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1169/2337] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:38:54.512380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.519525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:54.523532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf56a7-af0e-4035-82c3-a4be51307c7a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.530052 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.531161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.531504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.531973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.532680 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.534374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.547506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1169/2338] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:38:54.551981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.563539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d959c-db81-4727-b4fc-a416ce5749b9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.569048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.569672 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.569944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.570219 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.570859 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.572516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57da133-0e67-46b6-9d8f-bfbd11e77785 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.585799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1170/2339] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:54.590352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.601279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f126ec-987c-4d7a-9679-720320c6373b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.606344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.607030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.607292 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.607553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] PATH_INFO: `/v3/projects/6ac6082b0e22434a98ed261262c754f1/users/4adee55c1b534f04b0ba094df880edba/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.623436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac6082b0e22434a98ed261262c754f1, user_id=4adee55c1b534f04b0ba094df880edba, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.626036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.641635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1170/2340] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:54 2022] PUT /identity/v3/projects/6ac6082b0e22434a98ed261262c754f1/users/4adee55c1b534f04b0ba094df880edba/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:54.652093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.652412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.652828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.683720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] MFA Rules not processed for user `4adee55c1b534f04b0ba094df880edba`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:54.685063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:54.747629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:54.757772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1171/2341] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:54.759127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.825401 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1aeee-f6e1-4232-988d-3a4523b03511 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.830065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.830689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.830963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.831228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.831861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.834045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.865511 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1171/2342] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:38:54.872187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.887843 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc198e9-ceb9-4edc-b4e4-0095ccb4da3c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:54.955261 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:54.956040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:54.956362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:54.956626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:54.957270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:54.959255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:54.986310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1172/2343] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:38:54 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:38:54.987551 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:54.998568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28efca-d290-4f87-8cc6-1f7d8ecf70b6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.003440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.004043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.004347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.004611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.005276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:55.006853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:55.020613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1172/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:55.025598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:55.036210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3d257-2157-47d7-8254-5fbe5d857b17 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.041554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.042198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.042468 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.042735 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] PATH_INFO: `/v3/projects/8737dcaeea9446db87522d968659c29c/users/1c13177aab3d4956a6fed1c20924ea43/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.058946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8737dcaeea9446db87522d968659c29c, user_id=1c13177aab3d4956a6fed1c20924ea43, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:55.061819 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:55.077538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1173/2345] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:55 2022] PUT /identity/v3/projects/8737dcaeea9446db87522d968659c29c/users/1c13177aab3d4956a6fed1c20924ea43/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:38:55.081829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:55.091868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb45c-1e1b-4af2-ba3f-2f417069d761 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.151534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.152203 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.152510 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.152758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.153439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:55.155061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92ffb2-76d6-45a9-8503-d54e3c44f164 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:55.168534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1173/2346] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:38:55.170019 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:55.181067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940bd5-1ece-4788-9e2a-5b5507dc659d admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.186283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.186952 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.187227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.187494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] PATH_INFO: `/v3/projects/8737dcaeea9446db87522d968659c29c/users/1c13177aab3d4956a6fed1c20924ea43/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.189765 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8737dcaeea9446db87522d968659c29c, user_id=1c13177aab3d4956a6fed1c20924ea43, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:55.192576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:55.205957 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1174/2347] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:38:55 2022] PUT /identity/v3/projects/8737dcaeea9446db87522d968659c29c/users/1c13177aab3d4956a6fed1c20924ea43/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:38:55.217761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.218175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.218549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.248313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] MFA Rules not processed for user `1c13177aab3d4956a6fed1c20924ea43`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:38:55.249638 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:55.280734 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:38:55.322655 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1174/2348] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:38:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:38:55.334303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:55.336139 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:38:55.400970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c20440-359e-4c94-a562-f9744107e81f admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.405542 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.406185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.406445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.406719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:55.418739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:55.420243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:55.430441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1175/2349] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:38:55.967049 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:38:55.977558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555f4d9-0a45-4f0b-afd2-7fdb36bb342a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:38:55.981871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:38:55.982614 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:38:55.982977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:38:55.983252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:38:56.046737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:38:56.048260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:38:56.058180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1175/2350] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:38:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:00.147631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:00.149218 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:00.161712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48599b-8a1c-4689-90cb-82eaae9976f5 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:00.165694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:00.166284 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:00.166555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:00.166823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:00.215338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:00.225412 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:00.226954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e515f74-9ab3-4e4f-91be-dec9a975e221 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:00.236064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1176/2351] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:02.349637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:02.419193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4d0e8-f70c-4071-8f49-38cfd7c0eb75 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:02.425079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:02.425731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:02.426009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:02.426271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] PATH_INFO: `/v3/users/b59655017cc547c5b47b698b9a0cbf21` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:02.427537 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59655017cc547c5b47b698b9a0cbf21)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:02.429536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:02.479448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59655017cc547c5b47b698b9a0cbf21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:02.486254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59655017cc547c5b47b698b9a0cbf21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:02.493335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:02.500693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59655017cc547c5b47b698b9a0cbf21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:02.501600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59655017cc547c5b47b698b9a0cbf21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:02.510556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59655017cc547c5b47b698b9a0cbf21'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:02.516371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1176/2352] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:02 2022] DELETE /identity/v3/users/b59655017cc547c5b47b698b9a0cbf21 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:02.521156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:02.521660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:02.521772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:02.541644 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:02.542896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:02.610471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1177/2353] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:02.620644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:02.622285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:02.683465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38906873-c737-4be5-88f2-f67258d4fcd4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:02.687847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:02.688497 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:02.688776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:02.689030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:02.698932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:02.700414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:02.711949 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1177/2354] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:02.981121 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:02.982683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:02.998414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2924b-12cd-416c-9eed-b57693519a96 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:03.005206 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:03.005871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:03.006343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:03.006527 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] PATH_INFO: `/v3/projects/6435d0cb59c34bd48e559477b1a02ea0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:03.007686 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6435d0cb59c34bd48e559477b1a02ea0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:03.009970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:03.059358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6435d0cb59c34bd48e559477b1a02ea0', 'request_id': 'req-3b75cc2d-49e1-43ea-9811-d846147f297b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.065647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6435d0cb59c34bd48e559477b1a02ea0', 'request_id': 'req-3b75cc2d-49e1-43ea-9811-d846147f297b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.077004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6435d0cb59c34bd48e559477b1a02ea0', 'request_id': 'req-3b75cc2d-49e1-43ea-9811-d846147f297b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.088474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] The token cache is being invalidate because project 6435d0cb59c34bd48e559477b1a02ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:03.088764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.089431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.090979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1178/2355] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:02 2022] DELETE /identity/v3/projects/6435d0cb59c34bd48e559477b1a02ea0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:03.095165 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:03.106282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e241fd-1b08-4171-81cd-2151cd6d137a service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:03.168451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:03.169095 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:03.169376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:03.169648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] PATH_INFO: `/v3/users/4adee55c1b534f04b0ba094df880edba` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:03.170889 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adee55c1b534f04b0ba094df880edba)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:03.172904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:03.227972 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adee55c1b534f04b0ba094df880edba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.234920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adee55c1b534f04b0ba094df880edba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.246656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adee55c1b534f04b0ba094df880edba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.247357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adee55c1b534f04b0ba094df880edba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.255803 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adee55c1b534f04b0ba094df880edba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.262159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1178/2356] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:03 2022] DELETE /identity/v3/users/4adee55c1b534f04b0ba094df880edba => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:03.551229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:03.552885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:03.566884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75cc2d-49e1-43ea-9811-d846147f297b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:03.635228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:03.635827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:03.636101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:03.636400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] PATH_INFO: `/v3/projects/6ac6082b0e22434a98ed261262c754f1` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:03.637517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac6082b0e22434a98ed261262c754f1)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:03.639438 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:03.656505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:03.688165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac6082b0e22434a98ed261262c754f1', 'request_id': 'req-90508a1e-be25-48ff-9293-24cf251ae80a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.694328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac6082b0e22434a98ed261262c754f1', 'request_id': 'req-90508a1e-be25-48ff-9293-24cf251ae80a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.702738 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac6082b0e22434a98ed261262c754f1', 'request_id': 'req-90508a1e-be25-48ff-9293-24cf251ae80a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.713173 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] The token cache is being invalidate because project 6ac6082b0e22434a98ed261262c754f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:03.713458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.714088 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.715556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1179/2357] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:03 2022] DELETE /identity/v3/projects/6ac6082b0e22434a98ed261262c754f1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:03.719805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:03.721260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:03.730464 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:03.734330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad10ea5-003e-4b4f-a551-ff29e6ea0d6e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:03.794148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:03.794787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:03.795062 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:03.795327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] PATH_INFO: `/v3/users/1c13177aab3d4956a6fed1c20924ea43` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:03.796473 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c13177aab3d4956a6fed1c20924ea43)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:03.798443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:03.846263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c13177aab3d4956a6fed1c20924ea43'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.853360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c13177aab3d4956a6fed1c20924ea43'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.865119 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c13177aab3d4956a6fed1c20924ea43'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.865775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c13177aab3d4956a6fed1c20924ea43'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.874114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-69e1da58-d916-4e54-accf-dfe91c87b575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c13177aab3d4956a6fed1c20924ea43'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:03.880393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1179/2358] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:03 2022] DELETE /identity/v3/users/1c13177aab3d4956a6fed1c20924ea43 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:04.154954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.157257 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:04.173607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508a1e-be25-48ff-9293-24cf251ae80a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.244880 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.245548 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.245934 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.246210 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] PATH_INFO: `/v3/projects/8737dcaeea9446db87522d968659c29c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.247541 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8737dcaeea9446db87522d968659c29c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.249499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.323381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8737dcaeea9446db87522d968659c29c', 'request_id': 'req-929bb080-618f-4b35-916a-0e147940a0a0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:04.330258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8737dcaeea9446db87522d968659c29c', 'request_id': 'req-929bb080-618f-4b35-916a-0e147940a0a0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:04.340560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8737dcaeea9446db87522d968659c29c', 'request_id': 'req-929bb080-618f-4b35-916a-0e147940a0a0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:04.351276 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] The token cache is being invalidate because project 8737dcaeea9446db87522d968659c29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:04.351560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:04.352252 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-929bb080-618f-4b35-916a-0e147940a0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:04.353775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1180/2359] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:04 2022] DELETE /identity/v3/projects/8737dcaeea9446db87522d968659c29c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:04.359953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.360260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.360527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.381103 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:04.382339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:04.445184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1180/2360] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:04.453892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.454171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.454440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.475512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:04.476848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:04.485786 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1181/2361] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:04.490069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.500096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16746c-69b2-4d17-b8a5-3acda2d62918 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.506043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.506648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.506938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.507189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.507842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:04.508148 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.510118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.522407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1181/2362] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:04.526842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.536474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87771870-56c9-426a-8965-eedf1ea1e30c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.541410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.542035 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.542299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.542555 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.543181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.545108 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.577730 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1182/2363] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:04.582298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.593170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373d66-e750-4750-82e9-a8cc4c4a0a78 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.601950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:04.656421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.656980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.657258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.657528 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.658161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.660198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.681264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1182/2364] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:39:04.685614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.696669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291113b2-4319-47c8-85fa-c0dacc8bb519 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.702107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.702715 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.703047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.703248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.703918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.705667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.707204 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:04.717771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1183/2365] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:04.722468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.732909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0720c-6e7e-4a6d-a0e2-ea6f7fbdfecc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.738724 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.739332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.739609 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.739882 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.740568 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.742175 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d044ad-2df3-4583-847d-e6769b7cd9d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.753356 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1183/2366] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:04.763336 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:04.774255 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c21c-d309-421b-83b6-691ef77d4536 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:04.779307 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:04.779958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.780269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.780540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] PATH_INFO: `/v3/projects/d5a5ab60cfe04f7eae0dee99ab21d9e1/users/f810e26a59944ee3a9c5f56f5b1b29b4/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.796638 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a5ab60cfe04f7eae0dee99ab21d9e1, user_id=f810e26a59944ee3a9c5f56f5b1b29b4, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:04.799282 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:04.815383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1184/2367] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:04 2022] PUT /identity/v3/projects/d5a5ab60cfe04f7eae0dee99ab21d9e1/users/f810e26a59944ee3a9c5f56f5b1b29b4/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:04.827130 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927bf7-d264-4295-b458-301354874776 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:04.827416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927bf7-d264-4295-b458-301354874776 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:04.827686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927bf7-d264-4295-b458-301354874776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:04.857509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7c927bf7-d264-4295-b458-301354874776 None None] MFA Rules not processed for user `f810e26a59944ee3a9c5f56f5b1b29b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:04.858726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7c927bf7-d264-4295-b458-301354874776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:04.934763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1184/2368] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:04.940496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.007303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08edef2-c8e7-4e94-b5d6-485ee0c7abe7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.012768 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.013388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.013666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.013936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.014574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.016565 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.049543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1185/2369] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:04 2022] POST /identity/v3/projects => generated 409 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:05.051112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c927bf7-d264-4295-b458-301354874776 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.062824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c927bf7-d264-4295-b458-301354874776 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.122459 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.123085 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.123361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.123631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.124317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.126305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.150371 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1185/2370] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:05 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:05.153788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.165275 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cd25-4abb-4a48-9086-227f531369c0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.170592 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.171216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.171471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.171743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.172599 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.174187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.185159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1186/2371] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:05.189266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.199580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de733-17c1-4a74-88a5-2bef3af310ee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.204643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.205242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.205520 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.205789 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] PATH_INFO: `/v3/projects/864869dda21346eabb4e7d5b8ebe6e98/users/9ebe8f74cb294f56be5748f98eeb2ae0/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.208958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:05.222197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864869dda21346eabb4e7d5b8ebe6e98, user_id=9ebe8f74cb294f56be5748f98eeb2ae0, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.224885 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.240865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1186/2372] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:05 2022] PUT /identity/v3/projects/864869dda21346eabb4e7d5b8ebe6e98/users/9ebe8f74cb294f56be5748f98eeb2ae0/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:05.244900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.254977 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc80c-d4d7-414d-94b9-8e9e944617ff admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.311973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.312904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.313237 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.313570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.314631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.317249 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed5987-0455-4813-8096-cab90f90f7cc admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.331052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1187/2373] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:05.335252 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.346661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033dea5-8922-476a-8166-53b8a8ff0b04 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.351902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.352792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.353317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.353690 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] PATH_INFO: `/v3/projects/864869dda21346eabb4e7d5b8ebe6e98/users/9ebe8f74cb294f56be5748f98eeb2ae0/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.357933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864869dda21346eabb4e7d5b8ebe6e98, user_id=9ebe8f74cb294f56be5748f98eeb2ae0, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.360994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.378438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1187/2374] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:05 2022] PUT /identity/v3/projects/864869dda21346eabb4e7d5b8ebe6e98/users/9ebe8f74cb294f56be5748f98eeb2ae0/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:05.385097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.385416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.385739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.419150 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] MFA Rules not processed for user `9ebe8f74cb294f56be5748f98eeb2ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:05.420401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:05.452564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:05.493915 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1188/2375] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:05.504186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.506128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:05.567797 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700ec54-e358-45f4-ba13-3ba48313f6a6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.571767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.572422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.572697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.573006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.583079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.584723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58480f28-3683-4c69-bff8-48ce054f58ce service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.595081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1188/2376] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:05.752458 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:05.763067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e63f3-94ff-42db-bb5b-425a5b42f3c2 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:05.766968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:05.767627 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:05.767907 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:05.768212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:05.826425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:05.827832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:05.836732 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1189/2377] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:05 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:13.973224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:13.973864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:13.974080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:14.000026 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] MFA Rules not processed for user `76d8100e29a24533a014b8d073e45ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:14.001745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:14.077463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1189/2378] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:14.434234 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:14.435777 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:14.492995 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb68a6-5fc9-408a-9853-c688440be8f9 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:14.496804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:14.497413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:14.497684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:14.497946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:14.499669 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:14.510585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:14.511981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:14.520898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1190/2379] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:15.036347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:15.038063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:15.127482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251abe4-e33e-4c7b-ba2c-d9db9aaea62f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:15.132677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:15.133293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:15.133565 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:15.133829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:15.136451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:15.149360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:15.150859 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a85c-8bc2-44fc-b497-86d647d28b2a service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:15.164106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1190/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:16.478966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:16.546193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96941f0b-9668-48aa-bb6b-fae413a04f83 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:16.550903 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:16.551535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:16.551836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:16.552104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] PATH_INFO: `/v3/users/f810e26a59944ee3a9c5f56f5b1b29b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:16.553432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f810e26a59944ee3a9c5f56f5b1b29b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:16.555329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:16.565464 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:16.605227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f810e26a59944ee3a9c5f56f5b1b29b4'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:16.605968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f810e26a59944ee3a9c5f56f5b1b29b4'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:16.618350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f810e26a59944ee3a9c5f56f5b1b29b4'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:16.627352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f810e26a59944ee3a9c5f56f5b1b29b4'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:16.627939 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f810e26a59944ee3a9c5f56f5b1b29b4'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:16.642305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1191/2381] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:16 2022] DELETE /identity/v3/users/f810e26a59944ee3a9c5f56f5b1b29b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:16.646758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:16.647325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:16.647325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:16.667952 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:16.669195 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:16.673350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:16.729922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1191/2382] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:16.741400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:16.743083 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:16.805780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84917fb-d1b1-4f14-a8c6-79fe99dc8149 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:16.809868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:16.810470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:16.810739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:16.811003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:16.820713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:16.822127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:16.831856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1192/2383] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:17.201499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:17.202969 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:17.218107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9967b4-e442-4ba1-96b9-520674a0e2cc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:17.226561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:17.227285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:17.227529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:17.227793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] PATH_INFO: `/v3/projects/d5a5ab60cfe04f7eae0dee99ab21d9e1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:17.229120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a5ab60cfe04f7eae0dee99ab21d9e1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:17.231031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:17.281276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a5ab60cfe04f7eae0dee99ab21d9e1', 'request_id': 'req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.287854 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a5ab60cfe04f7eae0dee99ab21d9e1', 'request_id': 'req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.295772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a5ab60cfe04f7eae0dee99ab21d9e1', 'request_id': 'req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.310515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] The token cache is being invalidate because project d5a5ab60cfe04f7eae0dee99ab21d9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:17.321169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.322344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.325018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1192/2384] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:17 2022] DELETE /identity/v3/projects/d5a5ab60cfe04f7eae0dee99ab21d9e1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:17.331281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:17.349401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7f09b-83c0-4c30-a15b-3dce9b93253e service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:17.408849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:17.409454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:17.409731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:17.410001 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] PATH_INFO: `/v3/users/9ebe8f74cb294f56be5748f98eeb2ae0` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:17.411250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebe8f74cb294f56be5748f98eeb2ae0)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:17.413223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:17.451947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:17.466479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebe8f74cb294f56be5748f98eeb2ae0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.467291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebe8f74cb294f56be5748f98eeb2ae0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.479958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebe8f74cb294f56be5748f98eeb2ae0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.489300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebe8f74cb294f56be5748f98eeb2ae0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.489974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2165a24d-2734-40f8-8e14-6972d8c99bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebe8f74cb294f56be5748f98eeb2ae0'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:17.502906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1193/2385] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:17 2022] DELETE /identity/v3/users/9ebe8f74cb294f56be5748f98eeb2ae0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:17.889448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:17.891498 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:17.907814 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b2d6-ab7d-4dbb-b6a9-8afdb58ed98e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:17.967421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:17.968063 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:17.968346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:17.968636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] PATH_INFO: `/v3/projects/864869dda21346eabb4e7d5b8ebe6e98` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:17.969780 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=864869dda21346eabb4e7d5b8ebe6e98)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:17.971684 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.010857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:18.022243 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864869dda21346eabb4e7d5b8ebe6e98', 'request_id': 'req-ff85a097-ea97-47f2-ba5d-fa36d0075196'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:18.028801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864869dda21346eabb4e7d5b8ebe6e98', 'request_id': 'req-ff85a097-ea97-47f2-ba5d-fa36d0075196'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:18.037268 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864869dda21346eabb4e7d5b8ebe6e98', 'request_id': 'req-ff85a097-ea97-47f2-ba5d-fa36d0075196'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:18.053887 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] The token cache is being invalidate because project 864869dda21346eabb4e7d5b8ebe6e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:18.054260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:18.054925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ff85a097-ea97-47f2-ba5d-fa36d0075196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:18.056828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1193/2386] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:17 2022] DELETE /identity/v3/projects/864869dda21346eabb4e7d5b8ebe6e98 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:18.062195 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.062505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.063224 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.085030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:18.086305 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:18.166165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1194/2387] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:18.167975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.168297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.168567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.203723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:18.205128 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:18.215828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1194/2388] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:18.219787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.238899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34462c-0829-4036-a8bf-c3f85c57c615 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.246455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.247212 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.247549 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.248271 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.249050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:18.249359 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.251004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.279841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1195/2389] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:18.285180 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.295170 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bd514-6abf-4a5d-b454-bb01bb477f65 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.300133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.300805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.301080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.301350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.302126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.304092 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.335801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1195/2390] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:18.340264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.351921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875ba1d-181c-48a0-96bd-57126b384548 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.419338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.420205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.420552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.420967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.421805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.423935 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.456231 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1196/2391] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:39:18.457386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.468440 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1b28-513f-455e-be5f-45802cadca17 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.473014 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.473653 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.473930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.474202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.474850 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.476603 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.487813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1196/2392] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:18.493201 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.504713 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4102f8d-08c2-4601-8272-3a1fd9de6b08 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.510616 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.511239 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.511520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.511793 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.512479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.514104 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8da9e-fb46-420e-8d6f-004506065ad1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.525785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1197/2393] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:18.529863 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.541392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0962-5f49-47cc-a2e1-ad5d0a7b56c8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.546031 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.546688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.546961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.547227 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] PATH_INFO: `/v3/projects/a29ca1b3296949808369e8fe0eb7bd63/users/5cdccf40d5b74f988b92d248015ec202/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.562652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29ca1b3296949808369e8fe0eb7bd63, user_id=5cdccf40d5b74f988b92d248015ec202, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.565376 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.587733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1197/2394] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:18 2022] PUT /identity/v3/projects/a29ca1b3296949808369e8fe0eb7bd63/users/5cdccf40d5b74f988b92d248015ec202/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:18.597987 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.598265 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.598563 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.630388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] MFA Rules not processed for user `5cdccf40d5b74f988b92d248015ec202`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:18.631680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:18.636810 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:18.702537 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1198/2395] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:18.707059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.771620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d52e8-c47c-416e-9fef-7920be13ca2d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.777040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.777685 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.777963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.778233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.778898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.780897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.815199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1198/2396] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:18.816600 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.831539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f52129-6d39-40e7-9bba-561f1ded7e85 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.899368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.899929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.900193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.900462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.901099 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.903047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.928096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1199/2397] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:18 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:18.931647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.943369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342dbc-b319-42ce-9373-f06536c6eeaf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.949367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.950069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.950416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.950778 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.951539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.953288 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:18.970438 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1199/2398] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:18.974545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:18.985580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09972f0-4355-4e5f-90b8-0fb2243e1407 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:18.990649 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:18.991283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:18.992015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:18.992314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:18.992973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:18.994712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36fd4d-2013-48c4-aca0-1bc5f651a677 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.009325 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1200/2399] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:19.010973 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.037552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a50b3f-6263-4909-a8ac-535c265a8936 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.046605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.047416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.047771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.048752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] PATH_INFO: `/v3/projects/fbfa9147a87f40b18a3aa8462683e834/users/0eacb956ee004f7aba91f31e4e5dd014/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.065378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbfa9147a87f40b18a3aa8462683e834, user_id=0eacb956ee004f7aba91f31e4e5dd014, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.068812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.086601 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1200/2400] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:19 2022] PUT /identity/v3/projects/fbfa9147a87f40b18a3aa8462683e834/users/0eacb956ee004f7aba91f31e4e5dd014/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:39:19.160946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.160946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.161247 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.193946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] MFA Rules not processed for user `0eacb956ee004f7aba91f31e4e5dd014`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:19.195161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:19.269897 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1201/2401] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:19.274909 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.287257 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:19.374571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d235c5-c4b0-4814-b394-13d8ca9e7a0d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.388450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.390622 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.391830 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.392134 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.395073 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.400753 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.490043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1201/2402] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:19 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:19.491198 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.511472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbac963-34f0-4fa0-9e73-1de99519d13c None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.525333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:19.593731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.594628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.595233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.595700 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.596607 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.598829 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.628347 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1202/2403] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:19 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:19.633942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.647793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089703c-f833-4416-a2e0-ab5a12cd4546 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.656835 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.657958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.658256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.658585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.659436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.662298 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.679829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1202/2404] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:19.683985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.714825 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a066dd6-c52e-412f-ad37-71af73b5bf9b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.722134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.723416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.723996 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.725943 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] PATH_INFO: `/v3/projects/0918b59d761b4300936601293c74400a/users/dbafde897ae1473fb1b6a2f10a75c64f/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.750904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0918b59d761b4300936601293c74400a, user_id=dbafde897ae1473fb1b6a2f10a75c64f, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.757032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.790253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1203/2405] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:19 2022] PUT /identity/v3/projects/0918b59d761b4300936601293c74400a/users/dbafde897ae1473fb1b6a2f10a75c64f/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:19.794153 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.807229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54948409-153d-4511-abca-1a1c898f67fd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.924363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.925732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.926009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.926355 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.927111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.928787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691dc11-196f-4904-80ec-8d7dd02113fd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.943396 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1203/2406] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:19.944695 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:19.955812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea563-2767-4823-8b52-06f983c7c543 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:19.960747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:19.961546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.961900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.962233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] PATH_INFO: `/v3/projects/0918b59d761b4300936601293c74400a/users/dbafde897ae1473fb1b6a2f10a75c64f/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:19.964288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0918b59d761b4300936601293c74400a, user_id=dbafde897ae1473fb1b6a2f10a75c64f, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:19.966989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:19.978604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1204/2407] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:19 2022] PUT /identity/v3/projects/0918b59d761b4300936601293c74400a/users/dbafde897ae1473fb1b6a2f10a75c64f/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:19.989771 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:19.990129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:19.990332 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:20.029282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] MFA Rules not processed for user `dbafde897ae1473fb1b6a2f10a75c64f`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:20.030812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:20.125366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1204/2408] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:20.139543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:20.142011 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:20.291366 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40226-e7c7-4aea-a4bb-6033e01f6cd0 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:20.298946 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:20.300622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:20.301253 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:20.301823 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:20.315704 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:20.317828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:20.333611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1205/2409] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:20.393605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:20.395061 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:20.413672 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:20.419542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e429-c682-4450-a1f1-7fc758961f49 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:20.432997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:20.433641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:20.434075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:20.434374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:20.453927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:20.456002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:20.532161 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:20.533997 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa686c-9efe-43e2-a6e8-4ee794870ced service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:20.545437 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1205/2410] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:20.584268 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316757d4-a9d9-491b-af1a-edcead04ab8f service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:20.597910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:20.598707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:20.599056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:20.599400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:20.601466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:20.604924 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:20.695550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:20.697357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e5f78-d11e-4d6c-af91-1dbd1372ef26 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:20.742026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1206/2411] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:39:21.669487 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:21.670077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:21.670077 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:21.671647 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1206/2412] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:39:21.689462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848153-51c6-42da-a219-670aa898fd40 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:21.690044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848153-51c6-42da-a219-670aa898fd40 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:21.690044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52848153-51c6-42da-a219-670aa898fd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:21.731494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-52848153-51c6-42da-a219-670aa898fd40 None None] MFA Rules not processed for user `b07bb14ad0a147fda168a9165a74b7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:21.732956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-52848153-51c6-42da-a219-670aa898fd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:21.792574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1207/2413] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:39:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:21.810265 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:21.811925 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:21.825039 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a41d4-dae5-4803-bef7-5437cf07cf9c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:21.828515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:21.829116 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:21.829390 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:21.829656 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:21.836545 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:21.837962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db619-0195-4604-a9dc-ba964c8d3e1d service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:21.846021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1207/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:21.930844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:21.931102 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:21.931403 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] PATH_INFO: `/` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:21.932875 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1208/2415] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 11 13:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 13:39:21.938584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:21.938868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:21.939142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:21.979389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] MFA Rules not processed for user `edc1cd61b66547b2a388b1ead24a47c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:21.980750 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:21.991327 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1208/2416] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 11 13:39:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:22.004569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:22.006955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:22.023270 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1222747-0cc8-4d27-8df1-89d28dd7edb4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:22.027749 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:22.028421 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:22.028716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:22.029026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:22.040584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:22.042025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1858e4-f763-4db4-bb31-73f2d0e20905 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:22.054784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1209/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:23.643172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:23.714277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc87252-4b76-4bef-8a03-56907fa09cc4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:23.718697 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:23.719337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:23.719617 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:23.719890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] PATH_INFO: `/v3/users/5cdccf40d5b74f988b92d248015ec202` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:23.721115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cdccf40d5b74f988b92d248015ec202)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:23.723046 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:23.781467 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cdccf40d5b74f988b92d248015ec202'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:23.789832 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cdccf40d5b74f988b92d248015ec202'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:23.803366 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:23.806679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cdccf40d5b74f988b92d248015ec202'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:23.807434 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cdccf40d5b74f988b92d248015ec202'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:23.816842 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cdccf40d5b74f988b92d248015ec202'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:23.823743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1209/2418] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:23 2022] DELETE /identity/v3/users/5cdccf40d5b74f988b92d248015ec202 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:23.828014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:23.828559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:23.828688 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:23.849673 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:23.850931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:23.902352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:23.911248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1210/2419] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:23.921549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:23.923471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:23.983009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02934fd3-6139-4fd1-91bf-8df054b1b9de admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:23.986820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:23.987427 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:23.987696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:23.987957 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:23.997431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:23.998851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:24.008476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1210/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:24.358168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:24.359716 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:24.371931 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d17cd-a6f7-4896-8594-f6f3ab6e38fb None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:24.376512 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:24.377151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:24.377429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:24.377697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] PATH_INFO: `/v3/projects/a29ca1b3296949808369e8fe0eb7bd63` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:24.378805 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29ca1b3296949808369e8fe0eb7bd63)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:24.380729 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:24.429228 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29ca1b3296949808369e8fe0eb7bd63', 'request_id': 'req-1db900e5-58c7-4cb6-ba9e-470443834258'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.436942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29ca1b3296949808369e8fe0eb7bd63', 'request_id': 'req-1db900e5-58c7-4cb6-ba9e-470443834258'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.445333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29ca1b3296949808369e8fe0eb7bd63', 'request_id': 'req-1db900e5-58c7-4cb6-ba9e-470443834258'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.455904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] The token cache is being invalidate because project a29ca1b3296949808369e8fe0eb7bd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:24.456295 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.456967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.458679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1211/2421] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:24 2022] DELETE /identity/v3/projects/a29ca1b3296949808369e8fe0eb7bd63 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:24.462699 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:24.473122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df986cd-e563-40c9-a4f1-0fc16cc5a486 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:24.535625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:24.536344 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:24.536620 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:24.536888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] PATH_INFO: `/v3/users/0eacb956ee004f7aba91f31e4e5dd014` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:24.538120 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eacb956ee004f7aba91f31e4e5dd014)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:24.540102 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:24.595431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eacb956ee004f7aba91f31e4e5dd014'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.605580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eacb956ee004f7aba91f31e4e5dd014'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.619812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eacb956ee004f7aba91f31e4e5dd014'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.620736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eacb956ee004f7aba91f31e4e5dd014'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.629215 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eacb956ee004f7aba91f31e4e5dd014'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:24.635767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1211/2422] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:24 2022] DELETE /identity/v3/users/0eacb956ee004f7aba91f31e4e5dd014 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:24.907914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:24.909461 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:24.922543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db900e5-58c7-4cb6-ba9e-470443834258 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:24.986572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:24.987128 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:24.987400 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:24.987663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] PATH_INFO: `/v3/projects/fbfa9147a87f40b18a3aa8462683e834` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:24.988813 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbfa9147a87f40b18a3aa8462683e834)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:24.990739 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.040279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbfa9147a87f40b18a3aa8462683e834', 'request_id': 'req-40db5d5e-5732-4ff9-901a-6d74e6866fdb'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.046551 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbfa9147a87f40b18a3aa8462683e834', 'request_id': 'req-40db5d5e-5732-4ff9-901a-6d74e6866fdb'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.055064 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbfa9147a87f40b18a3aa8462683e834', 'request_id': 'req-40db5d5e-5732-4ff9-901a-6d74e6866fdb'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.066054 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] The token cache is being invalidate because project fbfa9147a87f40b18a3aa8462683e834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:25.066341 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.067015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.068615 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1212/2423] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:24 2022] DELETE /identity/v3/projects/fbfa9147a87f40b18a3aa8462683e834 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:25.072962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:25.074407 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.088110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad837b7b-9d0b-4116-a5c8-a52d3ee4d4ca admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:25.097518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:25.149377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:25.150016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.150290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.150556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] PATH_INFO: `/v3/users/dbafde897ae1473fb1b6a2f10a75c64f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.151686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbafde897ae1473fb1b6a2f10a75c64f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:25.153648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.201962 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbafde897ae1473fb1b6a2f10a75c64f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.209864 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbafde897ae1473fb1b6a2f10a75c64f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.221812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbafde897ae1473fb1b6a2f10a75c64f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.222527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbafde897ae1473fb1b6a2f10a75c64f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.230654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbafde897ae1473fb1b6a2f10a75c64f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.236852 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1212/2424] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:25 2022] DELETE /identity/v3/users/dbafde897ae1473fb1b6a2f10a75c64f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:25.456397 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:25.458433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.474278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:25.530208 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db5d5e-5732-4ff9-901a-6d74e6866fdb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:25.581610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:25.583554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.594109 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:25.594804 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.595094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.595416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.597476 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.658593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a19190-e254-4399-89c2-2a400d1aa57a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:25.662920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:25.664290 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb11d1-f3f8-4bd3-beda-8993fefd585b service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.665190 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:25.665921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.666109 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.666379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] PATH_INFO: `/v3/projects/0918b59d761b4300936601293c74400a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.668300 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0918b59d761b4300936601293c74400a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:25.669413 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.677345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1213/2425] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:25.711030 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:25.724819 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0918b59d761b4300936601293c74400a', 'request_id': 'req-403d75d1-0dd5-49f2-aebd-c511529d662e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.733071 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0918b59d761b4300936601293c74400a', 'request_id': 'req-403d75d1-0dd5-49f2-aebd-c511529d662e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.743003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0918b59d761b4300936601293c74400a', 'request_id': 'req-403d75d1-0dd5-49f2-aebd-c511529d662e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.757524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] The token cache is being invalidate because project 0918b59d761b4300936601293c74400a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:25.757865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.758492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-403d75d1-0dd5-49f2-aebd-c511529d662e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:25.760633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1213/2426] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:25 2022] DELETE /identity/v3/projects/0918b59d761b4300936601293c74400a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:25.767118 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.767396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.767665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.789068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:25.790545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.856784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1214/2427] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:25.868515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.868515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.868723 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.889439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:25.890657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:25.898583 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1214/2428] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:25.903153 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:25.913772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3a272-8c36-4ff1-96fe-96d02320c959 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:25.919490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:25.920135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.920508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.920779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.921429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:25.921725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:25.923330 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.936089 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1215/2429] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:39:25.940806 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:25.950748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba7b5f-1a2f-479a-af08-4273c1592a94 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:25.955793 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:25.956457 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:25.956726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:25.956992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:25.957729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:25.959679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:25.988169 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1215/2430] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:25 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:25.993465 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:26.004063 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35dafe-26dc-42ce-8309-6a55690263d5 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:26.008609 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:26.063681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:26.064332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.064611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.064860 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.065503 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:26.067415 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:26.091782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1216/2431] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:25 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:39:26.095950 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:26.106317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921bbe-98e5-4d35-acd8-4c7d73ea9704 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:26.111363 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:26.111977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.112284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.112555 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.113201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:26.114874 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:26.125217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1216/2432] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:26.129546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:26.140790 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ab4e0-d61e-493f-be44-75733720d1c3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:26.145575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:26.146194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.146684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.147051 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.147705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:26.149460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2654997-b526-4ce5-9e61-6d3e71e79b53 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:26.160891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1217/2433] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:26.165310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:26.175319 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb1db-eed3-449a-a180-8ed456ee8681 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:26.180353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:26.181079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.181436 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.181767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] PATH_INFO: `/v3/projects/779aa6f0ec434a01ad27b34b3a0429c9/users/6106c33e88564a1fbfe622fcf37e2060/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.196493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] RBAC: Authorizing `identity:create_grant(project_id=779aa6f0ec434a01ad27b34b3a0429c9, user_id=6106c33e88564a1fbfe622fcf37e2060, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:26.199686 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:26.213325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1217/2434] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:26 2022] PUT /identity/v3/projects/779aa6f0ec434a01ad27b34b3a0429c9/users/6106c33e88564a1fbfe622fcf37e2060/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:26.224632 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.225067 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.225250 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.252697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] MFA Rules not processed for user `6106c33e88564a1fbfe622fcf37e2060`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:26.253831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:26.322248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1218/2435] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:26.333531 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:26.335507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:26.395420 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffff8a-0629-405d-9069-e7139eac038f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:26.399139 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:26.399752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:26.400036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:26.400329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:26.408719 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:26.410141 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:26.419055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1218/2436] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:28.939782 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:28.941372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:29.011177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300458b-d144-4ead-9c81-c466bf1ff919 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.015842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.016526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.016858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.017162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] PATH_INFO: `/v3/users/2fc3a4b9afd14409b3aef17f49aafd29` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.018274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc3a4b9afd14409b3aef17f49aafd29)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.020318 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.038308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:29.079056 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc3a4b9afd14409b3aef17f49aafd29'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.079814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc3a4b9afd14409b3aef17f49aafd29'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.094460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc3a4b9afd14409b3aef17f49aafd29'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.104098 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc3a4b9afd14409b3aef17f49aafd29'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.105107 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-65867d3e-4128-47c6-8f04-3945bd98d83b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc3a4b9afd14409b3aef17f49aafd29'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.121172 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1219/2437] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:28 2022] DELETE /identity/v3/users/2fc3a4b9afd14409b3aef17f49aafd29 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:29.309532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.323525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554f3b-0bb8-4c19-a0c8-09ad26cf23fa service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.388648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.389451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.389824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.390187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] PATH_INFO: `/v3/projects/659a7ca9a3684db4b6a29c780e7f8eb7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.391439 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=659a7ca9a3684db4b6a29c780e7f8eb7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.393581 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.395238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:29.445074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659a7ca9a3684db4b6a29c780e7f8eb7', 'request_id': 'req-c0143c36-e310-467a-bf47-1d1cce2fe0fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.451177 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659a7ca9a3684db4b6a29c780e7f8eb7', 'request_id': 'req-c0143c36-e310-467a-bf47-1d1cce2fe0fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.459006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659a7ca9a3684db4b6a29c780e7f8eb7', 'request_id': 'req-c0143c36-e310-467a-bf47-1d1cce2fe0fc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.469305 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] The token cache is being invalidate because project 659a7ca9a3684db4b6a29c780e7f8eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:29.469588 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.470210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0143c36-e310-467a-bf47-1d1cce2fe0fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:29.471706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1219/2438] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:29 2022] DELETE /identity/v3/projects/659a7ca9a3684db4b6a29c780e7f8eb7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:29.477077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.477394 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.477734 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.497747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:29.498963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:29.557297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1220/2439] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:29.565442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.565722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.565995 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.586453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:29.587652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:29.596005 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1220/2440] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:29.600425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.610361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d15804-79ab-49b1-88f3-61538ccc477b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.615060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.615778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.616057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.616361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.617046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:29.617351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.618956 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.630743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1221/2441] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:29.635310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.645410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac80f8f-5e1d-4059-9643-759da0a75eb1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.650760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.651399 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.651671 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.651932 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.652592 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.654536 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.687317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1221/2442] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:29 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:29.688572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.699580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17297eee-455e-4cf7-a565-002baf7d1790 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.755706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.756373 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.756647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.756914 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.757543 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.759422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.782787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1222/2443] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:29 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:29.787343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.798895 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e41f53-da3c-41f0-b480-c3eaead6490c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.805149 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.805840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.806263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.806606 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.807331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.809056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.819695 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1222/2444] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:29.824047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.834096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd207c-760d-43f0-9d10-59cbc227c4b6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.839094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.839892 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.840199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.840471 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.841138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.842884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04921dd3-222c-450f-8a07-d347118ed255 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.848410 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:29.855243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1223/2445] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:29.859295 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:29.869482 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013d1f-efc9-475a-8529-c6df2519709b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:29.874707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:29.875402 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.875673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.875938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] PATH_INFO: `/v3/projects/862926e6e5334f37aefe16a66778e5e2/users/43968ba640744091b997fdd15462fc1b/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.891866 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862926e6e5334f37aefe16a66778e5e2, user_id=43968ba640744091b997fdd15462fc1b, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:29.894857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:29.909324 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1223/2446] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:29 2022] PUT /identity/v3/projects/862926e6e5334f37aefe16a66778e5e2/users/43968ba640744091b997fdd15462fc1b/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:29.921454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:29.921742 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:29.922016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:29.951561 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] MFA Rules not processed for user `43968ba640744091b997fdd15462fc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:29.952824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:30.023661 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1224/2447] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:30.035625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:30.037679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:30.115722 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96687c42-dbd0-409d-a281-50953618f320 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:30.121788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:30.122394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:30.122668 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:30.122931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:30.130811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:30.132801 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c9ef1-aa61-4189-bf6c-287ef48b4b90 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:30.142947 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1224/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:35.028327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:35.105052 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12077387-1d61-4bd1-82d7-f34d730c6e64 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:35.109637 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:35.110850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.111132 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.111442 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] PATH_INFO: `/v3/users/43968ba640744091b997fdd15462fc1b` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.112612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43968ba640744091b997fdd15462fc1b)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:35.114518 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:35.162331 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43968ba640744091b997fdd15462fc1b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.162982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43968ba640744091b997fdd15462fc1b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.176238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43968ba640744091b997fdd15462fc1b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.185473 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43968ba640744091b997fdd15462fc1b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.186082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43968ba640744091b997fdd15462fc1b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.201105 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1225/2449] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:35 2022] DELETE /identity/v3/users/43968ba640744091b997fdd15462fc1b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:35.205818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.206317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.206317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.226425 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:35.227755 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:35.233818 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:35.296841 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1225/2450] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:35.307197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:35.308668 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:35.319432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:35.375887 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112c43-1854-4e0b-a6a2-9350d3005785 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:35.379795 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:35.380447 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.380724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.380992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.390788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:35.392248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af08879-8c9b-4cfc-9fb7-a1c99ed36d42 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:35.401759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1226/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:35.675558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:35.677025 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:35.695625 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3d84d-658d-4d25-a34b-6853583d1f6b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:35.701307 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:35.701858 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.702048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.702323 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] PATH_INFO: `/v3/projects/862926e6e5334f37aefe16a66778e5e2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.703489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862926e6e5334f37aefe16a66778e5e2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:35.705423 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:35.754204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862926e6e5334f37aefe16a66778e5e2', 'request_id': 'req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.761000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862926e6e5334f37aefe16a66778e5e2', 'request_id': 'req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.770491 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862926e6e5334f37aefe16a66778e5e2', 'request_id': 'req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.781126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] The token cache is being invalidate because project 862926e6e5334f37aefe16a66778e5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:35.781433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.782027 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14531e9a-e665-41ac-b49d-3ff4e1fa3cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:35.783494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1226/2452] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:35 2022] DELETE /identity/v3/projects/862926e6e5334f37aefe16a66778e5e2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:35.788523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.788807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.789077 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.810621 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:35.811844 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:35.872096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1227/2453] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:35.880185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.880469 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.880737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.901380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:35.902580 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:35.910920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1227/2454] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:35.915066 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:35.924964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1343bb5-1348-44f4-9b53-02fa37caa96d None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:35.929721 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:35.930324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.930606 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.930873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.931508 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:35.931831 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:35.933505 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:35.945194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1228/2455] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:35.949411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:35.958869 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936689e7-d325-4941-9db3-fb0120063c31 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:35.964461 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:35.965088 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:35.965357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:35.965618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:35.966276 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:35.968285 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:35.999294 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1228/2456] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:35 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:36.004726 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.015703 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73883-26d5-42f2-882e-4d64c036d129 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.073947 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.074604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.074811 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.075078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.075718 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.077629 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.100941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1229/2457] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:36.103923 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.113623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.114414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aca8a7-8f4f-4ba1-9fe3-be87663ea24c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.115564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.119207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.119910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.120202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.120475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.121118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.122708 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.127388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ca540-3fee-4c1d-a59d-d2e167407a59 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.132181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.132812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.132921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1229/2458] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:36.133074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.133348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] PATH_INFO: `/v3/users/6106c33e88564a1fbfe622fcf37e2060` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.134685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6106c33e88564a1fbfe622fcf37e2060)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.136967 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.138521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.149241 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b7ce-52e3-4077-8092-e5d4be7dcf17 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.154817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.155361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.155627 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.155898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.156589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.158194 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.162862 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:36.171082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1230/2459] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:36.176006 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.195081 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f1cef-91e9-466a-8b10-5a639163d2e3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.199658 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.200418 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.200748 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.200893 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] PATH_INFO: `/v3/projects/7670a786e0004262ae9f9829b5198b23/users/65071e2033dd40e2b33ff3c67ab9b58d/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.202891 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6106c33e88564a1fbfe622fcf37e2060'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:36.203973 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6106c33e88564a1fbfe622fcf37e2060'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:36.217935 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7670a786e0004262ae9f9829b5198b23, user_id=65071e2033dd40e2b33ff3c67ab9b58d, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.218501 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6106c33e88564a1fbfe622fcf37e2060'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:36.220648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b644fe-101a-43e9-b723-6b118972fb59 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.228573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6106c33e88564a1fbfe622fcf37e2060'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:36.229339 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9bdadb4a-bf47-4ac3-af79-4d05d05c4ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6106c33e88564a1fbfe622fcf37e2060'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:36.236320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1231/2460] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:36 2022] PUT /identity/v3/projects/7670a786e0004262ae9f9829b5198b23/users/65071e2033dd40e2b33ff3c67ab9b58d/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:36.243652 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1230/2461] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:36 2022] DELETE /identity/v3/users/6106c33e88564a1fbfe622fcf37e2060 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:36.248211 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.248451 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.248569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.249047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.249186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.249453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.270896 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:36.272350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.285385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] MFA Rules not processed for user `65071e2033dd40e2b33ff3c67ab9b58d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:36.287191 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.297178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:36.391827 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1232/2462] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:36.395898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1231/2463] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:36.396959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.401474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.406315 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.413140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.417834 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf13497-ac24-48a2-9825-6634b6412718 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.424001 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.424512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.425264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.425264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.426096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.429004 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.464683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1233/2464] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:36.476351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.489731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af2f3-06cf-489a-8a5a-fe2eea4cd895 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.559900 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4d70c-7ccf-4f6f-a0e3-24a1e407bc40 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.563775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.564433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.564714 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.564984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.565624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.567535 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.581817 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117194a-e580-4477-add9-170e225f339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.583004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117194a-e580-4477-add9-170e225f339c service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.583477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117194a-e580-4477-add9-170e225f339c service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.583772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117194a-e580-4477-add9-170e225f339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.601650 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1234/2465] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:36.605732 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.612598 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117194a-e580-4477-add9-170e225f339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.615549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e5d-4579-45ac-af3b-ad781d90a831 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.616731 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117194a-e580-4477-add9-170e225f339c service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.621329 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.622271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.622730 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.622846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:36.623277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.624256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.626624 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.630428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1232/2466] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:39:36.640652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1235/2467] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:36.642779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117194a-e580-4477-add9-170e225f339c service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.654306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117194a-e580-4477-add9-170e225f339c service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.658981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.659770 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.660047 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.660343 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] PATH_INFO: `/v3/projects/b09043b0abee479ca05c25561f4942fd/users/3475f9032fb74e82a70e5d5716c62b70/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.676901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09043b0abee479ca05c25561f4942fd, user_id=3475f9032fb74e82a70e5d5716c62b70, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.679764 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.694580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1233/2468] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:36 2022] PUT /identity/v3/projects/b09043b0abee479ca05c25561f4942fd/users/3475f9032fb74e82a70e5d5716c62b70/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:36.699167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.710142 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0217-518c-4e5e-adaa-8d0df06f6782 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.770267 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.770954 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.771238 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.771504 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.772176 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.773837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.784678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1236/2469] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:36.788964 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.806095 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75d04-ba47-4765-b9ee-81216c786c3e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.811166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.812071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.812406 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.812698 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] PATH_INFO: `/v3/projects/b09043b0abee479ca05c25561f4942fd/users/3475f9032fb74e82a70e5d5716c62b70/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.814775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09043b0abee479ca05c25561f4942fd, user_id=3475f9032fb74e82a70e5d5716c62b70, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:36.817614 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433df29-8c06-4fd0-8071-1decaec5277f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:36.829096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1234/2470] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:36 2022] PUT /identity/v3/projects/b09043b0abee479ca05c25561f4942fd/users/3475f9032fb74e82a70e5d5716c62b70/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:36.833433 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.838600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:36.840057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.840364 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.840639 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.872278 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] MFA Rules not processed for user `3475f9032fb74e82a70e5d5716c62b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:36.873684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.898414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023adf16-857d-4f4a-8750-e0a16b8485b5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:36.957678 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:36.958843 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:36.959543 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:36.960048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:36.962675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:36.970785 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1235/2471] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:36.984297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:36.985900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.054557 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e4abf-093a-4193-8131-34ef3ddfd730 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.058571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.059197 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.059477 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.059746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.071800 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.075127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.077040 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.078513 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.087049 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1236/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:39:37.088068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.089453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1237/2473] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:39:37.104454 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.163705 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.166589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:37.195428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610ff77-4aad-4253-a7c0-b2092e94064a service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.200483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.201124 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.201396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.201662 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] PATH_INFO: `/v3/projects/779aa6f0ec434a01ad27b34b3a0429c9` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.203151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] RBAC: Authorizing `identity:delete_project(project_id=779aa6f0ec434a01ad27b34b3a0429c9)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.204908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.246137 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b0bc1-f807-49b2-9cfb-fdcfd31ecdb7 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.252938 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.253629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.253912 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.254172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] PATH_INFO: `/v3/users/65071e2033dd40e2b33ff3c67ab9b58d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.255474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65071e2033dd40e2b33ff3c67ab9b58d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.257489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.263848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779aa6f0ec434a01ad27b34b3a0429c9', 'request_id': 'req-2ada5e1e-d14a-4a2c-b839-04107f3b856b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.269515 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:37.271832 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779aa6f0ec434a01ad27b34b3a0429c9', 'request_id': 'req-2ada5e1e-d14a-4a2c-b839-04107f3b856b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.281604 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779aa6f0ec434a01ad27b34b3a0429c9', 'request_id': 'req-2ada5e1e-d14a-4a2c-b839-04107f3b856b'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.296161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] The token cache is being invalidate because project 779aa6f0ec434a01ad27b34b3a0429c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:37.296455 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.297188 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-2ada5e1e-d14a-4a2c-b839-04107f3b856b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.298759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1237/2474] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:37 2022] DELETE /identity/v3/projects/779aa6f0ec434a01ad27b34b3a0429c9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:37.303681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237671c-26a4-4b3c-9fa3-b4a365ae7a99 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.303962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237671c-26a4-4b3c-9fa3-b4a365ae7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.304262 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237671c-26a4-4b3c-9fa3-b4a365ae7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.316270 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65071e2033dd40e2b33ff3c67ab9b58d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.324300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7237671c-26a4-4b3c-9fa3-b4a365ae7a99 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:37.325201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65071e2033dd40e2b33ff3c67ab9b58d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.325462 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7237671c-26a4-4b3c-9fa3-b4a365ae7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.337632 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65071e2033dd40e2b33ff3c67ab9b58d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.338373 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65071e2033dd40e2b33ff3c67ab9b58d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.347200 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87d4a9d7-9e51-48d3-8208-477537a74c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65071e2033dd40e2b33ff3c67ab9b58d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:37.356281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1238/2475] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:37 2022] DELETE /identity/v3/users/65071e2033dd40e2b33ff3c67ab9b58d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:37.359557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.359845 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.360956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.392193 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:37.393629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.420070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1238/2476] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:37.425856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.426226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.426575 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.460205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:37.461861 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.464369 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1239/2477] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:37.473402 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1239/2478] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:37.479515 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.479984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.481652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.491667 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbb81d-fe26-46fc-a135-1dd5e5386b7b None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.501902 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.502773 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.503288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.503647 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.504526 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:37.504830 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.506584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.521664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1240/2479] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:37.526185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.542845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6ca9f-41dc-4e5b-9ffb-2b2ec730d573 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.549828 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.551382 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.551890 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.552693 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.553436 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.555663 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.566331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66084f-c14d-4b37-9448-3efdaf226642 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.572101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.572984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.573258 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.573527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.593652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.595140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.609723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1241/2480] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:37.609933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1240/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:37.610747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.663989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:37.690453 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe616d8-7fd2-4f19-9e53-d47d88c4d565 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.696233 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.696906 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.697186 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.697459 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.698163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.700101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.730087 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1242/2482] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:37.734978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.745823 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72c45-9dbb-487a-b1b0-25dcb3ebeeb8 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.750667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.751284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.751554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.751817 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.752506 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.754110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.765361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1241/2483] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:37.769623 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.786936 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ec6b-d55e-49dc-bc7c-f98b56dbdd9a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.792309 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.793082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.793489 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.793761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.794413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.796288 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc0f2c-dec2-4100-8d56-009cc613a5b7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.807144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1243/2484] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:37.811443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.821472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3eea0-1697-4d0c-92a4-a14e8a0fbe67 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:37.826110 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:37.827916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.827916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.827916 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] PATH_INFO: `/v3/projects/67d45f22783f474184a77f566663054e/users/f37f54ef73544963af83a4446f66c9bf/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.844811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d45f22783f474184a77f566663054e, user_id=f37f54ef73544963af83a4446f66c9bf, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:37.847831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:37.851337 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:37.868207 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1242/2485] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:37 2022] PUT /identity/v3/projects/67d45f22783f474184a77f566663054e/users/f37f54ef73544963af83a4446f66c9bf/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:37.877564 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:37.877855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:37.878111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:37.909163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] MFA Rules not processed for user `f37f54ef73544963af83a4446f66c9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:37.910834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.927204 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:37.929829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:37.995573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1244/2486] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:38.007475 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:38.008942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.021445 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080603af-d9b1-42e7-b476-00291cede550 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:38.029185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:38.029838 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:38.030114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:38.030377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] PATH_INFO: `/v3/projects/7670a786e0004262ae9f9829b5198b23` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:38.031486 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7670a786e0004262ae9f9829b5198b23)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:38.033472 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:38.075314 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9e7ae-d087-44b6-bda4-3d3f170c5be4 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:38.084629 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7670a786e0004262ae9f9829b5198b23', 'request_id': 'req-4a22cbab-efbe-409e-9b99-9c5b708cfa57'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.092501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7670a786e0004262ae9f9829b5198b23', 'request_id': 'req-4a22cbab-efbe-409e-9b99-9c5b708cfa57'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.102878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7670a786e0004262ae9f9829b5198b23', 'request_id': 'req-4a22cbab-efbe-409e-9b99-9c5b708cfa57'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.115059 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] The token cache is being invalidate because project 7670a786e0004262ae9f9829b5198b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:38.115361 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.116075 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.117761 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1243/2487] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:37 2022] DELETE /identity/v3/projects/7670a786e0004262ae9f9829b5198b23 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:39:38.121879 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:38.123322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.137927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:38.138746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:38.139093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:38.139425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:38.141269 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.161529 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:38.196783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22cbab-efbe-409e-9b99-9c5b708cfa57 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:38.202199 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:38.202982 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:38.203210 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:38.203664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] PATH_INFO: `/v3/users/3475f9032fb74e82a70e5d5716c62b70` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:38.205475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3475f9032fb74e82a70e5d5716c62b70)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:38.208157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:38.208834 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:38.210419 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:38.223048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1245/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:39:38.239777 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:38.265468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3475f9032fb74e82a70e5d5716c62b70'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.274674 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3475f9032fb74e82a70e5d5716c62b70'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.290244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3475f9032fb74e82a70e5d5716c62b70'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.291009 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3475f9032fb74e82a70e5d5716c62b70'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.306136 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3475f9032fb74e82a70e5d5716c62b70'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.315689 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1244/2489] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:38 2022] DELETE /identity/v3/users/3475f9032fb74e82a70e5d5716c62b70 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:39:38.525113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:38.526719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.695516 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e605c5d-85c9-4cb9-93ba-cb29e99da387 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:38.780246 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:38.780472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:38.780581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:38.780850 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:38.783069 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.799191 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:38.801942 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66530f1c-d96b-4b2c-bc5b-191203e0b102 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:38.828626 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1246/2490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:38.837943 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:38.839977 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:38.920659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba5e1-be98-4368-ab50-bd5132a74f4a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:38.925784 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:38.926527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:38.926881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:38.927217 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] PATH_INFO: `/v3/projects/b09043b0abee479ca05c25561f4942fd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:38.928549 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09043b0abee479ca05c25561f4942fd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:38.930494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:38.983847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09043b0abee479ca05c25561f4942fd', 'request_id': 'req-1d4dbb6b-6341-40f1-9837-3a001c912258'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:38.993910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09043b0abee479ca05c25561f4942fd', 'request_id': 'req-1d4dbb6b-6341-40f1-9837-3a001c912258'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:39.007481 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09043b0abee479ca05c25561f4942fd', 'request_id': 'req-1d4dbb6b-6341-40f1-9837-3a001c912258'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:39.040275 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] The token cache is being invalidate because project b09043b0abee479ca05c25561f4942fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:39.040664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:39.041980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1d4dbb6b-6341-40f1-9837-3a001c912258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:39.044016 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1245/2491] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:38 2022] DELETE /identity/v3/projects/b09043b0abee479ca05c25561f4942fd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:39.049160 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.049572 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.051335 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.071389 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:39.072940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:39.117762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:39.147450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1247/2492] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:39.155443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.155762 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.156260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.180162 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:39.181523 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:39.196920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1246/2493] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:39.202802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.215131 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a438457-5baf-4b25-a8f6-541bbbdd28b6 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.220205 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.221348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.221348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.221348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.222351 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:39.222883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.225177 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.242664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1248/2494] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:39.243875 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.254530 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6fab3-8ab9-4802-b2a9-454e0821b43b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.259529 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.260589 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.260867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.261133 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.261837 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.264593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.303855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1247/2495] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:39.306940 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.323004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f907720-d897-4a6a-a660-830329e9de4a admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.390567 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.391216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.391491 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.391757 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.392434 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.395733 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.426851 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1249/2496] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:39.427939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.439271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93972d8-7d29-48ec-bbcd-dbf6dd046241 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.446597 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.447480 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.447810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.448078 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.448769 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.450411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.452246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:39.465788 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1248/2497] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:39.467362 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.477435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce9bc-f32a-420f-9523-770c97916fec admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.482134 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.482741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.483018 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.483283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.483928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.485553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053794b-c249-4c3d-a882-d09b46cf90c0 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.496354 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1250/2498] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:39.500766 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.511336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330865-9ac3-4910-bf1e-047ce3a39362 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.516221 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.516890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.517172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.517446 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] PATH_INFO: `/v3/projects/9db6551eb3374f16a09acb7288b83fcc/users/2aa17913be4d417d93054bf4191068ef/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.533126 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db6551eb3374f16a09acb7288b83fcc, user_id=2aa17913be4d417d93054bf4191068ef, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.535809 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.549877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1249/2499] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:39 2022] PUT /identity/v3/projects/9db6551eb3374f16a09acb7288b83fcc/users/2aa17913be4d417d93054bf4191068ef/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:39.562446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671cc4f-b2aa-4d77-bbc1-6ec37747a9de None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.562963 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671cc4f-b2aa-4d77-bbc1-6ec37747a9de None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.563260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671cc4f-b2aa-4d77-bbc1-6ec37747a9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.591717 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-8671cc4f-b2aa-4d77-bbc1-6ec37747a9de None None] MFA Rules not processed for user `2aa17913be4d417d93054bf4191068ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:39.592965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8671cc4f-b2aa-4d77-bbc1-6ec37747a9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:39.663583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1251/2500] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:39.664768 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.737379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610109da-3f41-42d7-bc5f-614014892620 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.737748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610109da-3f41-42d7-bc5f-614014892620 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.737961 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610109da-3f41-42d7-bc5f-614014892620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.744731 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfbbd-0933-40c0-87f9-770ea888bb3e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.751021 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.751775 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.752054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.752934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.753590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.755693 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.762112 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-610109da-3f41-42d7-bc5f-614014892620 None None] MFA Rules not processed for user `f37f54ef73544963af83a4446f66c9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:39.763353 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-610109da-3f41-42d7-bc5f-614014892620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:39.793322 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1250/2501] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:39:39.797379 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.817187 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:39.855610 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1252/2502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:39.897394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c94af-5853-4955-8a90-45e459ae0c79 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.902544 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.903262 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.903542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.903810 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.904488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.906460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.935919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1251/2503] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:39 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:39.939969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610109da-3f41-42d7-bc5f-614014892620 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.951778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610109da-3f41-42d7-bc5f-614014892620 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.956540 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.957157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.957440 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.957714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:39.958368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:39.960004 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:39.971000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1253/2504] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:39.975047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:39.986679 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf926f6-d296-4f1c-a318-05a2b8c27c0c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:39.991824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:39.992687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:39.992961 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:39.993354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] PATH_INFO: `/v3/projects/f3081aa9356948798e1da5e38145a606/users/cd114a95f5fd45159a68a1de4593234d/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.011853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3081aa9356948798e1da5e38145a606, user_id=cd114a95f5fd45159a68a1de4593234d, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.014736 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.032154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1252/2505] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:39 2022] PUT /identity/v3/projects/f3081aa9356948798e1da5e38145a606/users/cd114a95f5fd45159a68a1de4593234d/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:40.034595 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.045068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e8c99-66df-41d6-990b-36c38a878733 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.113388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:40.114036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.114334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.114611 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.115283 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.117494 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcaad5-a96e-41f7-99b7-39e208a095f2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.132836 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1254/2506] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:40.134273 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.147280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808de31-819d-48d0-be22-da499af24f9e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.152279 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:40.153314 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.153590 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.153861 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] PATH_INFO: `/v3/projects/f3081aa9356948798e1da5e38145a606/users/cd114a95f5fd45159a68a1de4593234d/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.155894 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3081aa9356948798e1da5e38145a606, user_id=cd114a95f5fd45159a68a1de4593234d, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.158829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.171310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1253/2507] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:40 2022] PUT /identity/v3/projects/f3081aa9356948798e1da5e38145a606/users/cd114a95f5fd45159a68a1de4593234d/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:39:40.181925 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.183399 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.183720 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.231706 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] MFA Rules not processed for user `cd114a95f5fd45159a68a1de4593234d`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:40.233582 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:40.335612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1255/2508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:40.349112 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.350820 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:40.364430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:40.422774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811c9f8-cfd2-4f8b-94b4-75be9d48e658 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.427904 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:40.428999 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.429325 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.429560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.439479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.441675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.452628 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1254/2509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:40.833653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.842792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769806-0276-4b59-a693-cbce2cc7dc54 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.847645 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd114a95f5fd45159a68a1de4593234d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3081aa9356948798e1da5e38145a606', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:40.848441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.848727 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.849003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.849750 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:40.850057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.851684 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.873586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1256/2510] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:40.874742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.884282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575bdb-7b1d-45be-909a-dd62a40b871b service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.890118 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd114a95f5fd45159a68a1de4593234d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3081aa9356948798e1da5e38145a606', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:40.890743 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:40.891024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:40.891293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:40.892847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:40.894802 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:40.921633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1255/2511] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:40 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:40.927080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:40.939794 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b35b5-e014-4fc1-bb2c-db4b906e9c2c tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:40.993158 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=100 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:41.019076 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd114a95f5fd45159a68a1de4593234d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3081aa9356948798e1da5e38145a606', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:41.019774 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:41.020078 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:41.020384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:41.021059 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:41.021352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:41.022955 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:41.038274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1257/2512] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:39:41.071554 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:41.082064 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54dac1-4d00-45ee-8030-a467bcfb15b0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:41.087411 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd114a95f5fd45159a68a1de4593234d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3081aa9356948798e1da5e38145a606', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:41.088107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:41.088479 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:41.088782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] PATH_INFO: `/v3/projects/caf6c71ca0d0413f974f1a6677843f1d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:41.090598 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorizing `identity:delete_project(project_id=caf6c71ca0d0413f974f1a6677843f1d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:41.093065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:41.148792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf6c71ca0d0413f974f1a6677843f1d', 'request_id': 'req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.156405 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf6c71ca0d0413f974f1a6677843f1d', 'request_id': 'req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.165098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf6c71ca0d0413f974f1a6677843f1d', 'request_id': 'req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.176020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] The token cache is being invalidate because project caf6c71ca0d0413f974f1a6677843f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:41.176353 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.177000 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0516458a-fd8b-4ef2-bc8c-88fff5ef3518 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.178816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1256/2513] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:41 2022] DELETE /identity/v3/projects/caf6c71ca0d0413f974f1a6677843f1d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:41.385921 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:41.387741 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:41.453539 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b4cf0-e8cc-4a4b-90d9-bea95a2f59f0 tempest-VolumeQuotaClassesTest-2019397169 tempest-VolumeQuotaClassesTest-2019397169-project-admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:41.510576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:41.511217 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:41.511496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:41.511767 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] PATH_INFO: `/v3/users/2aa17913be4d417d93054bf4191068ef` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:41.512981 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa17913be4d417d93054bf4191068ef)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:41.514926 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:41.563545 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa17913be4d417d93054bf4191068ef'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.564376 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa17913be4d417d93054bf4191068ef'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.580480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa17913be4d417d93054bf4191068ef'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.593772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa17913be4d417d93054bf4191068ef'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.594544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa17913be4d417d93054bf4191068ef'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:41.604294 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:41.609871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1258/2514] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:41 2022] DELETE /identity/v3/users/2aa17913be4d417d93054bf4191068ef => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:41.615068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:41.615897 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:41.616198 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:41.635984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:41.637246 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:41.703351 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1257/2515] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:41.713792 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:41.715681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:41.786003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a3c9d-3cbd-482a-af9b-7885485bab1e admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:41.790485 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:41.791553 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:41.791848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:41.792245 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:41.804009 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:41.805533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:41.817428 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1259/2516] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:42.149735 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:42.151408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:42.174146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a20e18-0c34-4017-97d3-7b822575d2e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:42.179745 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:42.180477 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:42.180892 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:42.181320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] PATH_INFO: `/v3/projects/9db6551eb3374f16a09acb7288b83fcc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:42.182980 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db6551eb3374f16a09acb7288b83fcc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:42.185747 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:42.195816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:42.238709 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db6551eb3374f16a09acb7288b83fcc', 'request_id': 'req-c0550566-7891-4993-acad-6dadbed134de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.245760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db6551eb3374f16a09acb7288b83fcc', 'request_id': 'req-c0550566-7891-4993-acad-6dadbed134de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.252946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db6551eb3374f16a09acb7288b83fcc', 'request_id': 'req-c0550566-7891-4993-acad-6dadbed134de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.264266 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] The token cache is being invalidate because project 9db6551eb3374f16a09acb7288b83fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:42.264559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.265182 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.266687 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1258/2517] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:42 2022] DELETE /identity/v3/projects/9db6551eb3374f16a09acb7288b83fcc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:42.270746 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:42.281017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e8a9-0619-4e05-820b-53ed2fd921a0 service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:42.340523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:42.341184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:42.341460 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:42.341724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] PATH_INFO: `/v3/users/cd114a95f5fd45159a68a1de4593234d` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:42.342846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd114a95f5fd45159a68a1de4593234d)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:42.344826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:42.397350 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd114a95f5fd45159a68a1de4593234d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.398042 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd114a95f5fd45159a68a1de4593234d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.409900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd114a95f5fd45159a68a1de4593234d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.419180 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd114a95f5fd45159a68a1de4593234d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.419809 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d89e471c-896e-491d-915d-1a3ae22d77e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd114a95f5fd45159a68a1de4593234d'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.432879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1260/2518] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:39:42 2022] DELETE /identity/v3/users/cd114a95f5fd45159a68a1de4593234d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:42.720844 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:42.723244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:42.738465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0550566-7891-4993-acad-6dadbed134de admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:42.814631 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:42.815442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:42.815993 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:42.816393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] PATH_INFO: `/v3/projects/f3081aa9356948798e1da5e38145a606` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:42.817698 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3081aa9356948798e1da5e38145a606)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:42.819872 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:42.876928 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3081aa9356948798e1da5e38145a606', 'request_id': 'req-4f2959e5-186d-48f0-b686-6d209a4560cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.884484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3081aa9356948798e1da5e38145a606', 'request_id': 'req-4f2959e5-186d-48f0-b686-6d209a4560cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.894534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3081aa9356948798e1da5e38145a606', 'request_id': 'req-4f2959e5-186d-48f0-b686-6d209a4560cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.909230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] The token cache is being invalidate because project f3081aa9356948798e1da5e38145a606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:39:42.909541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.910205 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4f2959e5-186d-48f0-b686-6d209a4560cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:39:42.912611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1259/2519] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:39:42 2022] DELETE /identity/v3/projects/f3081aa9356948798e1da5e38145a606 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:42.919017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:42.919493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:42.919941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:42.943784 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:42.945743 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:42.972413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:43.024260 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1261/2520] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:43.032098 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.032441 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.032717 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.052808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:43.054023 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:43.062209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1260/2521] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:43.066525 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.077220 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e8b20-8c06-4eb0-b38c-1811328cec71 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.081146 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.081842 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.082032 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.082306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.083280 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:39:43.084017 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.085226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.099329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1262/2522] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:39:43.104048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.114346 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4e64-dd3c-4cbd-ae75-3d2c99c57830 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.119488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.120309 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.120585 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.120855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.121509 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.123540 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.146393 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:43.159700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1261/2523] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:43 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:39:43.161413 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.171372 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e060-3024-4817-ba7f-617fecafdda3 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.231466 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.232144 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.232450 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.232714 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.233396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.235358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.259243 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1263/2524] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:43 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:43.264358 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.274584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83389b48-f8a9-4562-99ae-a0410dba686c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.279284 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.279958 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.280278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.280546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.281197 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.282811 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.293524 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1262/2525] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 13:39:43.297585 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.308045 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d4981-cc9c-42b9-8497-f0e925d3ec0c admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.314689 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.315312 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.315589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.315856 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.316550 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.318168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66b263-e156-43a8-99b0-a281acc6ce2f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.330879 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1264/2526] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:43.335354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.346074 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0481-f401-4e5b-b506-3473318e9eb7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.351209 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.351921 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.352219 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.352495 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] PATH_INFO: `/v3/projects/4637de297087426ebc3e2dfe2ae9e43e/users/6f2c2cd58c6444a6b182f72f522aba2a/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.368458 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4637de297087426ebc3e2dfe2ae9e43e, user_id=6f2c2cd58c6444a6b182f72f522aba2a, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.371157 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.386124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1263/2527] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:43 2022] PUT /identity/v3/projects/4637de297087426ebc3e2dfe2ae9e43e/users/6f2c2cd58c6444a6b182f72f522aba2a/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:43.391287 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307d05-f831-44ff-b22f-ce57b2582200 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.391571 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307d05-f831-44ff-b22f-ce57b2582200 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.391864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307d05-f831-44ff-b22f-ce57b2582200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.411664 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-20307d05-f831-44ff-b22f-ce57b2582200 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:43.413320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-20307d05-f831-44ff-b22f-ce57b2582200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:43.490130 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1265/2528] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:43.499851 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:43.501815 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:43.563010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1b9d6-9181-418d-8b42-49b51acc8b46 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:43.566633 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:43.567256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:43.567604 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:43.567929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:43.577665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:43.579144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:43.589552 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1264/2529] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:44.228691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.228969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.229238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.257871 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] MFA Rules not processed for user `6f2c2cd58c6444a6b182f72f522aba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:44.258928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:44.280675 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:44.332849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1266/2530] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:44.337471 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.347463 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9d1a8-79db-41b7-8039-f98c4baf3c82 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.351856 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.352494 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.352772 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.353043 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.353675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.355626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.382813 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1265/2531] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:39:44 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:44.387193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.398185 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9339564-6cd4-4bbe-a5cd-264d3d7274fa None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.455913 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.456636 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.456855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.457122 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.457753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.459665 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.483048 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1267/2532] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:39:44 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:39:44.487377 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.498178 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d1151-41d2-43b6-994e-c874e9421106 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.503256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.503868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.504172 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.504449 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.505101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.506721 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.518385 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1266/2533] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:44.522945 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.533628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448fc7-54d5-4102-84f7-fc6119e63287 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.538681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.539345 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.539657 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.539929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] PATH_INFO: `/v3/projects/f63ed988d73f47ee8b318d6d12b767bc/users/7a539a1250a040c2aedf0d63d977b732/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.555125 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63ed988d73f47ee8b318d6d12b767bc, user_id=7a539a1250a040c2aedf0d63d977b732, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.557818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.572352 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1268/2534] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:44 2022] PUT /identity/v3/projects/f63ed988d73f47ee8b318d6d12b767bc/users/7a539a1250a040c2aedf0d63d977b732/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:44.576657 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.586783 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54ef26-9c3e-47a1-9758-499e12b21f8d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.643245 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.643770 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.644047 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.644354 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.644994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.646591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5d993-91a9-4f73-9dc2-723b13cf22ad admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.657626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1267/2535] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:39:44.664050 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:44.675826 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3bc3-5dd0-4ad8-af94-12d903753ea2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:44.680941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:44.681605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:44.681884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:44.682155 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] PATH_INFO: `/v3/projects/f63ed988d73f47ee8b318d6d12b767bc/users/7a539a1250a040c2aedf0d63d977b732/roles/5ae4cdd2d721472191d8c3e071059abe` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:44.684248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63ed988d73f47ee8b318d6d12b767bc, user_id=7a539a1250a040c2aedf0d63d977b732, role_id=5ae4cdd2d721472191d8c3e071059abe)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:44.686908 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:44.698014 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1269/2536] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:39:44 2022] PUT /identity/v3/projects/f63ed988d73f47ee8b318d6d12b767bc/users/7a539a1250a040c2aedf0d63d977b732/roles/5ae4cdd2d721472191d8c3e071059abe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:39:45.357293 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:45.358048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:45.358048 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:45.387799 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] MFA Rules not processed for user `7a539a1250a040c2aedf0d63d977b732`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:45.389382 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:45.408673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:45.477776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1268/2537] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:45.489463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:45.491223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:45.550097 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59cdc5-655a-4fa3-8c87-d56c2bb184ba admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:45.554161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:45.554876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:45.555157 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:45.555424 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:45.581214 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:45.615506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:45.616974 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b458a9-c7e0-431e-abde-433600ee24dc service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:45.625941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1270/2538] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:47.003501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:47.005089 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:47.018941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb34e07-f979-4711-843a-4f5d593d8840 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:47.022959 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:47.023560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:47.023831 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:47.024096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:47.033503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:47.034922 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:47.044812 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1269/2539] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:39:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:39:55.622193 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=16 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:57.040442 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=27 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:39:57.613379 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e26e0-d70d-45e8-8dbd-e82aaa6e7c81 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:57.613864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e26e0-d70d-45e8-8dbd-e82aaa6e7c81 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:57.614072 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e26e0-d70d-45e8-8dbd-e82aaa6e7c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:57.670547 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-134e26e0-d70d-45e8-8dbd-e82aaa6e7c81 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:57.671888 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-134e26e0-d70d-45e8-8dbd-e82aaa6e7c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:57.740920 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1271/2540] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:39:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:57.790380 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:57.791853 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:57.886069 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532296b-f12d-4852-8b95-59374d209550 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:57.892310 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:57.892839 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:57.893121 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:57.893392 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:57.903416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:57.905087 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:57.913776 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1270/2541] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:39:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:39:58.442074 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efda8-df31-4bf9-abf6-269306106408 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:58.442463 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efda8-df31-4bf9-abf6-269306106408 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:58.442694 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4efda8-df31-4bf9-abf6-269306106408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:58.462566 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-1f4efda8-df31-4bf9-abf6-269306106408 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:58.463712 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-1f4efda8-df31-4bf9-abf6-269306106408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:58.472479 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1272/2542] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:58.488024 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:58.499155 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ff116-0ead-4361-9394-1e8d170c6be4 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:58.504643 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:58.505247 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:58.505526 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:58.505796 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:58.514430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:58.516035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:58.524240 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1271/2543] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:39:58.763869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c46e-458b-4fbd-983c-381d06cdb85e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:58.764381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c46e-458b-4fbd-983c-381d06cdb85e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:58.764493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c46e-458b-4fbd-983c-381d06cdb85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:58.789367 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-b005c46e-458b-4fbd-983c-381d06cdb85e None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:58.790630 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-b005c46e-458b-4fbd-983c-381d06cdb85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:58.798759 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1273/2544] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 11 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:39:58.812716 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:58.824825 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986f33c-31ed-41ea-a8ca-80e135c34c08 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:58.830566 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:58.831186 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:58.831484 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:58.831816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:58.840038 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:58.841664 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:58.849865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1272/2545] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:39:59.184509 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:59.185445 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:59.185791 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:59.207433 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] MFA Rules not processed for user `70eff2211f2741b1b3fc3f9d0c660d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:39:59.208723 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:39:59.217111 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1274/2546] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 11 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:39:59.229865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:39:59.241236 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87df0-5660-4420-b107-4b5e3a91b161 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:39:59.246522 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f89a9124ef147f9a402e7df844ed088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:39:59.247168 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:39:59.247453 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:39:59.247915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:39:59.256683 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:39:59.258290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:39:59.267239 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1273/2547] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 11 13:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 13:40:00.357573 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:00.423845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8004f-3a8d-485c-a032-84144d725991 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:00.428333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:00.429010 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:00.429293 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:00.429569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] PATH_INFO: `/v3/users/6f2c2cd58c6444a6b182f72f522aba2a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:00.430711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2c2cd58c6444a6b182f72f522aba2a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:00.432679 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:00.481589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2c2cd58c6444a6b182f72f522aba2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.482236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2c2cd58c6444a6b182f72f522aba2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.487439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:00.498100 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2c2cd58c6444a6b182f72f522aba2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.505927 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2c2cd58c6444a6b182f72f522aba2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.506559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2c2cd58c6444a6b182f72f522aba2a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.520580 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1275/2548] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:00 2022] DELETE /identity/v3/users/6f2c2cd58c6444a6b182f72f522aba2a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:00.696154 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:00.715308 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67418-e45a-4524-a773-a170ef151fd2 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:00.776936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:00.777474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:00.777752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:00.778055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] PATH_INFO: `/v3/projects/4637de297087426ebc3e2dfe2ae9e43e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:00.779237 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4637de297087426ebc3e2dfe2ae9e43e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:00.781213 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:00.819729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:00.833297 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4637de297087426ebc3e2dfe2ae9e43e', 'request_id': 'req-8278c4e6-12da-499d-b30b-af83b3847826'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.840080 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4637de297087426ebc3e2dfe2ae9e43e', 'request_id': 'req-8278c4e6-12da-499d-b30b-af83b3847826'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.849878 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4637de297087426ebc3e2dfe2ae9e43e', 'request_id': 'req-8278c4e6-12da-499d-b30b-af83b3847826'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.865146 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] The token cache is being invalidate because project 4637de297087426ebc3e2dfe2ae9e43e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:00.865584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.866454 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:00.868263 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1274/2549] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:00 2022] DELETE /identity/v3/projects/4637de297087426ebc3e2dfe2ae9e43e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:00.873181 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:00.875388 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:00.889840 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fabb1-4ef7-4c3d-b6b6-b19bdb6f1ec9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:00.947882 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:00.948544 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:00.948790 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:00.949084 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] PATH_INFO: `/v3/users/7a539a1250a040c2aedf0d63d977b732` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:00.950226 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a539a1250a040c2aedf0d63d977b732)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:00.952138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.001772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a539a1250a040c2aedf0d63d977b732'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.002443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a539a1250a040c2aedf0d63d977b732'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.015190 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a539a1250a040c2aedf0d63d977b732'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.024474 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a539a1250a040c2aedf0d63d977b732'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.025159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a539a1250a040c2aedf0d63d977b732'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.038625 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1276/2550] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:00 2022] DELETE /identity/v3/users/7a539a1250a040c2aedf0d63d977b732 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:01.416654 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.418201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.431206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278c4e6-12da-499d-b30b-af83b3847826 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.461574 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.463145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.502053 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.502707 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.502979 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.503242 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] PATH_INFO: `/v3/users/f37f54ef73544963af83a4446f66c9bf` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.504499 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37f54ef73544963af83a4446f66c9bf)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.506430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.523036 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:01.542232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac718e-dc45-4389-bf48-7dc5ae9bdafb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.548202 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.549133 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.549578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.549918 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] PATH_INFO: `/v3/projects/f63ed988d73f47ee8b318d6d12b767bc` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.551030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63ed988d73f47ee8b318d6d12b767bc)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.553068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.557115 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37f54ef73544963af83a4446f66c9bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.564002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37f54ef73544963af83a4446f66c9bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.578519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37f54ef73544963af83a4446f66c9bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.579233 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37f54ef73544963af83a4446f66c9bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.587065 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-653f6c6a-a349-40a3-9ea3-710abbfdb8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37f54ef73544963af83a4446f66c9bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.593877 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1275/2551] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:01 2022] DELETE /identity/v3/users/f37f54ef73544963af83a4446f66c9bf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:01.598244 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.598600 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.598870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.603683 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63ed988d73f47ee8b318d6d12b767bc', 'request_id': 'req-9d2cfad2-1edc-42e3-a70e-2a76925f778f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.610135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63ed988d73f47ee8b318d6d12b767bc', 'request_id': 'req-9d2cfad2-1edc-42e3-a70e-2a76925f778f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.619214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:01.619446 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63ed988d73f47ee8b318d6d12b767bc', 'request_id': 'req-9d2cfad2-1edc-42e3-a70e-2a76925f778f'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.621764 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.631357 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] The token cache is being invalidate because project f63ed988d73f47ee8b318d6d12b767bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:01.631643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.632323 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-9d2cfad2-1edc-42e3-a70e-2a76925f778f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:01.633816 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1277/2552] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:01 2022] DELETE /identity/v3/projects/f63ed988d73f47ee8b318d6d12b767bc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:40:01.639384 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291870cc-9ff0-4e45-a113-258e8a5db628 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.639605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291870cc-9ff0-4e45-a113-258e8a5db628 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.639877 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291870cc-9ff0-4e45-a113-258e8a5db628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.655140 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:01.659523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-291870cc-9ff0-4e45-a113-258e8a5db628 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:01.660868 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-291870cc-9ff0-4e45-a113-258e8a5db628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.682634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1276/2553] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:01.694253 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.696302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.720324 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1278/2554] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:40:01.729578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.729869 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.730145 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.752140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:01.753432 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.754041 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc37cc-6bf6-416f-ba24-11b097e1c3ff None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.757703 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.758335 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.758665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.758867 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.762839 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1279/2555] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:40:01.767082 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.768610 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.770097 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.776467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42de24f-8bce-40a0-9414-9d4d6f012085 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.779448 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1277/2556] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:40:01.781979 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.782725 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.783016 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.783322 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.784003 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:40:01.784348 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.786299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.800171 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1280/2557] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 13:40:01.803964 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.814507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fae543-af38-4d5d-a811-f705a44a32ae service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.819634 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.820271 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.820547 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.820808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.821519 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.823431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.853596 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1278/2558] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:40:01 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 13:40:01.857517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.867705 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2f7c-4569-43be-b786-be427946c85b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.927381 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.928000 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.928320 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.928586 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.929227 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.931248 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.944258 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:01.957574 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.958039 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1281/2559] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:40:01 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:40:01.961425 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:01.962527 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:01.973327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b904e6-5ae7-49f7-a079-a743cbbc8dae admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.978330 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99b20-ec70-4d61-a07e-336e6abe2a8e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:01.980761 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.981396 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.981680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.981960 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.982778 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.983011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:01.983649 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:01.983920 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:01.984211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] PATH_INFO: `/v3/projects/67d45f22783f474184a77f566663054e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:01.984328 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.985340 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d45f22783f474184a77f566663054e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:01.987301 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:01.999216 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1282/2560] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:02.000496 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:02.010724 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0678651-68b6-429d-8279-c8b649b529a2 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:02.015876 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:02.016950 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.017327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.017653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.018277 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:02.020020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:02.030968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1283/2561] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:02.034710 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:02.038711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d45f22783f474184a77f566663054e', 'request_id': 'req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.045339 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d45f22783f474184a77f566663054e', 'request_id': 'req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.045568 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7f87-5d06-4bae-819f-d2da804dfdb6 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:02.052782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d45f22783f474184a77f566663054e', 'request_id': 'req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.063779 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] The token cache is being invalidate because project 67d45f22783f474184a77f566663054e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:02.064068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.064726 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8b3c4f56-e3f3-4fa6-86b0-8ebaf313dbdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.066636 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1279/2562] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:01 2022] DELETE /identity/v3/projects/67d45f22783f474184a77f566663054e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:02.107299 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:02.107989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.108303 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.108570 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] PATH_INFO: `/v3/projects/7c4d2540959240bfb234570cc6b0df7c/users/7e635972c90d4710b0fce147128788be/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.125719 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4d2540959240bfb234570cc6b0df7c, user_id=7e635972c90d4710b0fce147128788be, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:02.128538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:02.144691 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1284/2563] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:40:02 2022] PUT /identity/v3/projects/7c4d2540959240bfb234570cc6b0df7c/users/7e635972c90d4710b0fce147128788be/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:40:02.155824 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.156044 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.156304 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.186986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] MFA Rules not processed for user `7e635972c90d4710b0fce147128788be`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:02.188224 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:02.223975 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:02.262114 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1280/2564] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:02.274070 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:02.275965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:02.342736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf470d05-62a3-4540-b5cb-7fe0d648e8bb admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:02.347163 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:02.347812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.348321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.348597 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.357511 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:02.358941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363c9b9-4e94-4ae2-90ac-f410f3c9713c service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:02.367982 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1285/2565] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:02.388915 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:02.390715 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:02.461742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa73bd-d7ff-4062-b2c1-15dbcb7f2872 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:02.467206 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:02.467840 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.468132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.468403 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] PATH_INFO: `/v3/users/7e635972c90d4710b0fce147128788be` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.469641 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e635972c90d4710b0fce147128788be)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:02.471567 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:02.519430 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e635972c90d4710b0fce147128788be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.526381 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e635972c90d4710b0fce147128788be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.538978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e635972c90d4710b0fce147128788be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.539733 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e635972c90d4710b0fce147128788be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.547741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e635972c90d4710b0fce147128788be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:02.553836 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1281/2566] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:02 2022] DELETE /identity/v3/users/7e635972c90d4710b0fce147128788be => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:02.558483 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.558980 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.559092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.578864 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:02.580091 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:02.630306 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:02.643983 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1286/2567] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:02.654211 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:02.655774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:02.714652 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ea18-c9ff-41ae-9bba-a259bbf7a04b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:02.718615 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:02.719229 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:02.719501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:02.719763 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:02.726929 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:02.729688 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:02.731488 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8274ab3-6651-46d4-bb5e-b4261135c0df service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:02.744003 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1282/2568] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:03.030748 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.032329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:03.045696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca444498-9b74-43e8-89e1-63a1db245168 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.050975 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.051685 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.051962 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.052334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] PATH_INFO: `/v3/projects/7c4d2540959240bfb234570cc6b0df7c` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.053493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4d2540959240bfb234570cc6b0df7c)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.055584 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.105416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4d2540959240bfb234570cc6b0df7c', 'request_id': 'req-5d0add8d-efa3-483c-8d85-197fda9c4b6c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:03.112751 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4d2540959240bfb234570cc6b0df7c', 'request_id': 'req-5d0add8d-efa3-483c-8d85-197fda9c4b6c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:03.121416 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4d2540959240bfb234570cc6b0df7c', 'request_id': 'req-5d0add8d-efa3-483c-8d85-197fda9c4b6c'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:03.132304 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] The token cache is being invalidate because project 7c4d2540959240bfb234570cc6b0df7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:03.132583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:03.133209 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-5d0add8d-efa3-483c-8d85-197fda9c4b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:03.134755 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1287/2569] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:03 2022] DELETE /identity/v3/projects/7c4d2540959240bfb234570cc6b0df7c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:03.140277 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.140561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.140828 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.161742 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:03.162941 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:03.222431 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1283/2570] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:03.230577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.230801 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.231068 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.251660 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:03.252941 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:03.263073 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1288/2571] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:40:03.267489 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.277727 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3cb3e-8fb2-4c16-88b1-0bb62107fa5c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.283264 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.283899 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.284189 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.284474 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.285124 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:40:03.285422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.287079 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.298696 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1284/2572] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:40:03.302873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.313680 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46dcb3-7adb-4965-ac24-a7a25609ad49 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.320506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.321135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.321404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.321655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] PATH_INFO: `/v3/projects` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.322289 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.324251 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.354910 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1289/2573] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:40:03 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:40:03.359302 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.369737 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e321-2284-4c5d-b829-f4984d6ea9bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.429099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.429706 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.429984 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.430251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.430876 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.432857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.455936 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1285/2574] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:40:03 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:40:03.460266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.470803 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686f7d-b823-4212-bcb9-254f5d701454 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.476061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.476697 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.476968 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.477229 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.477874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.479467 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.489923 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1290/2575] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:03.493659 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.503930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a97a-1921-4147-8842-426c1d7990a9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.509898 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.510855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.511132 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.511394 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.512037 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.513661 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac23fc-fc96-4b57-ab9b-dd45655506b8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.524808 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1286/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:03.528989 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.539308 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3201-73ff-4e2b-890b-3fafc0ecb2be admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.545168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.545815 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.546086 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.546340 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] PATH_INFO: `/v3/projects/a935d51efdcb47fabc92f5b45c710c59/users/e8d3e675e5ac4549818a035fb99f3d93/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.564837 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a935d51efdcb47fabc92f5b45c710c59, user_id=e8d3e675e5ac4549818a035fb99f3d93, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.571997 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.586789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1291/2577] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:40:03 2022] PUT /identity/v3/projects/a935d51efdcb47fabc92f5b45c710c59/users/e8d3e675e5ac4549818a035fb99f3d93/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:03.656350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.656758 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.657226 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.694584 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] MFA Rules not processed for user `e8d3e675e5ac4549818a035fb99f3d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:03.695833 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:03.744460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:03.771807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1287/2578] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:03.783002 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:03.784904 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:03.847022 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a308-baf2-4082-9e54-8f2cb5963a97 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:03.852058 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:03.852687 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:03.852965 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:03.853232 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:03.861857 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:03.863281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:03.875355 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1292/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:13.767289 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:13.869165 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:21.841150 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:21.843020 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:21.915296 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403e7cb-0c99-4817-8500-76a06f3e9900 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:21.921521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:21.922135 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:21.922424 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:21.923260 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:21.925318 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:21.937320 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:21.938919 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a88b-583d-4fd3-bf61-e364196bc742 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:21.949855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1288/2580] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:25.456874 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:25.521985 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2fb-5366-41f5-b580-3c0137f7c081 service cinder] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:25.526560 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:25.527166 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:25.527443 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:25.527711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] PATH_INFO: `/v3/users/e8d3e675e5ac4549818a035fb99f3d93` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:25.528886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d3e675e5ac4549818a035fb99f3d93)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:25.530776 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:25.582618 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d3e675e5ac4549818a035fb99f3d93'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:25.583358 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d3e675e5ac4549818a035fb99f3d93'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:25.596499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d3e675e5ac4549818a035fb99f3d93'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:25.605215 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d3e675e5ac4549818a035fb99f3d93'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:25.605846 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d3e675e5ac4549818a035fb99f3d93'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:25.618199 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1293/2581] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:25 2022] DELETE /identity/v3/users/e8d3e675e5ac4549818a035fb99f3d93 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:25.622807 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:25.623360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:25.623360 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:25.642868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:25.644159 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:25.705196 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1289/2582] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:25.715383 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:25.716944 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:25.783470 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b123e-4243-422a-af6c-5293883314e9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:25.790138 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:25.791031 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:25.791407 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:25.791772 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:25.804431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:25.805886 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766bff2-f336-4490-9e12-4781fd97c824 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:25.815788 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1294/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:26.109127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.110677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:26.128468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c03ca-00eb-413c-9c60-fb88082abb59 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.133700 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.134357 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.134635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.134917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] PATH_INFO: `/v3/projects/a935d51efdcb47fabc92f5b45c710c59` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.136421 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a935d51efdcb47fabc92f5b45c710c59)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.138398 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.188559 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a935d51efdcb47fabc92f5b45c710c59', 'request_id': 'req-845bf76e-dc95-4fff-90aa-01971233c145'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:26.194970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a935d51efdcb47fabc92f5b45c710c59', 'request_id': 'req-845bf76e-dc95-4fff-90aa-01971233c145'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:26.203990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a935d51efdcb47fabc92f5b45c710c59', 'request_id': 'req-845bf76e-dc95-4fff-90aa-01971233c145'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:26.214571 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] The token cache is being invalidate because project a935d51efdcb47fabc92f5b45c710c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:26.214857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:26.215490 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-845bf76e-dc95-4fff-90aa-01971233c145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:26.217036 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1290/2584] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:26 2022] DELETE /identity/v3/projects/a935d51efdcb47fabc92f5b45c710c59 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:26.222116 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.222429 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.222707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.243300 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:26.244556 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:26.262928 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:26.309137 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1295/2585] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 13:40:26.317143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.317460 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.317760 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.338933 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:26.340158 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:26.349412 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1291/2586] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:26.354338 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.367753 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a1dfe-aec8-4bd4-8e04-6cbb96d7ac2e None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.374162 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.374787 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.375071 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.375329 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.375992 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81370) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 13:40:26.376478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.378113 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.391027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1296/2587] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 11 13:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 13:40:26.395557 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.405173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73693-deb4-4a61-abfd-91e197f943c2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.410414 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.411011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.411280 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.411546 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.412174 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.414101 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.418613 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:26.445888 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1292/2588] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 11 13:40:26 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:40:26.451737 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.464060 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4a5e-c166-4942-bc1f-95225b92ead9 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.521883 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.522490 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.522762 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.523026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] PATH_INFO: `/v3/users` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.523643 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.525569 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.552559 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1297/2589] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 11 13:40:26 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 13:40:26.556710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.568143 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf58c9-1771-42b9-b86b-ebeb502b3749 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.573593 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.574214 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.574492 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.574759 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.575410 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.577082 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.588045 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1293/2590] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:26.592365 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.603274 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c2ade-e4b2-4e41-973a-3740108ec36b admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.608094 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.608747 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.609025 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.609297 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.609972 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.611581 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1268be4-cf85-440e-85c0-3a00eba61b0c admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.622486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1298/2591] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 11 13:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 13:40:26.626578 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.637225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263c60-0700-4117-8c05-3cfc64f07e46 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.642317 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.642978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.643290 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.643532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] PATH_INFO: `/v3/projects/47058ad11d4a4ebd89fce84b8a37ed0e/users/627db7ad37654539b3528e3d61705421/roles/d121dc77eff143a186cdd79b2ce341b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.659532 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47058ad11d4a4ebd89fce84b8a37ed0e, user_id=627db7ad37654539b3528e3d61705421, role_id=d121dc77eff143a186cdd79b2ce341b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.662230 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.675978 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1294/2592] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 11 13:40:26 2022] PUT /identity/v3/projects/47058ad11d4a4ebd89fce84b8a37ed0e/users/627db7ad37654539b3528e3d61705421/roles/d121dc77eff143a186cdd79b2ce341b4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:26.687841 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.688096 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.688404 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.718814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] MFA Rules not processed for user `627db7ad37654539b3528e3d61705421`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:26.720037 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:26.786374 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1299/2593] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:26.798056 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:26.800173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:26.859010 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cdd01-51b8-4a40-a185-d6e952730d6f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:26.862913 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:26.863521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:26.863794 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:26.864067 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:26.872644 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:26.874068 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e62674-d7be-40cb-91a5-08735ce374ac service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:26.882865 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1295/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:26 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:40:35.239057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:35.258969 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:35.304240 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bed59-579e-41ba-851f-5a623a4cc969 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:35.309520 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:35.310168 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:35.310435 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:35.310707 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] PATH_INFO: `/v3/users/627db7ad37654539b3528e3d61705421` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:35.311873 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627db7ad37654539b3528e3d61705421)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:35.313845 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:35.363310 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627db7ad37654539b3528e3d61705421'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.364015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627db7ad37654539b3528e3d61705421'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.376441 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627db7ad37654539b3528e3d61705421'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.386327 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627db7ad37654539b3528e3d61705421'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.387101 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627db7ad37654539b3528e3d61705421'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.399319 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1300/2595] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:40:35 2022] DELETE /identity/v3/users/627db7ad37654539b3528e3d61705421 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:35.404374 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:35.404884 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:35.404986 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:35.426333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] MFA Rules not processed for user `c24104e3c5bf48be865c100865a4abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:40:35.427542 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:35.487503 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1296/2596] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:40:35.497620 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:35.499140 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:35.559401 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33622fd3-668e-400c-a4ec-4bf93986fff4 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:35.564264 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:35.564881 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:35.565281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:35.565583 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:35.575333 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:35.576855 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:35.586587 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1301/2597] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:40:35.876476 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:40:35.877970 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:40:35.892106 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ae8dc-fe02-4458-9520-1088db4a6a23 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:40:35.897553 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:40:35.898251 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:40:35.898507 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:40:35.898787 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] PATH_INFO: `/v3/projects/47058ad11d4a4ebd89fce84b8a37ed0e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:40:35.900002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47058ad11d4a4ebd89fce84b8a37ed0e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:40:35.902201 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:40:35.948738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:35.950054 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47058ad11d4a4ebd89fce84b8a37ed0e', 'request_id': 'req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.957939 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47058ad11d4a4ebd89fce84b8a37ed0e', 'request_id': 'req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.966493 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47058ad11d4a4ebd89fce84b8a37ed0e', 'request_id': 'req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.981903 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] The token cache is being invalidate because project 47058ad11d4a4ebd89fce84b8a37ed0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:40:35.982202 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.982953 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:40:35.984637 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1297/2598] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:40:35 2022] DELETE /identity/v3/projects/47058ad11d4a4ebd89fce84b8a37ed0e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:40:45.582554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:40:45.978665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:41:32.695298 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:32.697156 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:32.711170 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f7d8-7d03-446c-9780-acb25470d60a service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:32.763439 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:32.764057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:32.764361 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:32.764628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:32.766812 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:32.837589 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:32.839023 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:32.849577 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1302/2599] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:41:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:41:37.588313 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:37.589917 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:37.602963 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dffaf-e3a8-4d65-8b36-4ebdd8d949c5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:37.606868 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:37.607468 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:37.607740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:37.608008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:37.609855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:37.673501 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:37.674937 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:37.684648 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1298/2600] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:41:42.844326 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=48 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:41:43.985641 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:43.987452 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:44.002279 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89377ad-1db3-4b43-89f2-c326646bbafd service neutron] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:44.006900 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:44.007552 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:44.007827 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:44.008092 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] PATH_INFO: `/v3/users/983c742ffffe4a82a722eb1b1c23f971` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:44.009375 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=983c742ffffe4a82a722eb1b1c23f971)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:44.011666 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:44.063151 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983c742ffffe4a82a722eb1b1c23f971'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.063802 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983c742ffffe4a82a722eb1b1c23f971'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.077062 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983c742ffffe4a82a722eb1b1c23f971'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.085898 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983c742ffffe4a82a722eb1b1c23f971'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.086528 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983c742ffffe4a82a722eb1b1c23f971'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.098681 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1303/2601] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:41:43 2022] DELETE /identity/v3/users/983c742ffffe4a82a722eb1b1c23f971 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:41:44.277782 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:44.289911 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995252-6bbb-4b57-a849-9d8d9c8b94a1 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:44.353605 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:44.354254 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:44.354534 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:44.354805 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] PATH_INFO: `/v3/projects/81282f5bd6b34d90afaba281d5bbc16b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:44.355981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=81282f5bd6b34d90afaba281d5bbc16b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:44.357931 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:44.407183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81282f5bd6b34d90afaba281d5bbc16b', 'request_id': 'req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.417767 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81282f5bd6b34d90afaba281d5bbc16b', 'request_id': 'req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.429675 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81282f5bd6b34d90afaba281d5bbc16b', 'request_id': 'req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.440416 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] The token cache is being invalidate because project 81282f5bd6b34d90afaba281d5bbc16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:41:44.440702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.441336 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.442890 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1299/2602] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:41:44 2022] DELETE /identity/v3/projects/81282f5bd6b34d90afaba281d5bbc16b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:41:44.447117 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:44.448650 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:44.462236 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b8bf3-a9af-4ee9-b89d-eaa070b51caf admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:44.522238 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:44.522798 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:44.523080 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:44.523349 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] PATH_INFO: `/v3/users/41e18bc015b545acaa92588c84622a5a` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:44.524631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e18bc015b545acaa92588c84622a5a)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:44.526554 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:44.579281 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e18bc015b545acaa92588c84622a5a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.579970 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e18bc015b545acaa92588c84622a5a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.592824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e18bc015b545acaa92588c84622a5a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.601884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e18bc015b545acaa92588c84622a5a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.602722 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e18bc015b545acaa92588c84622a5a'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.605122 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:41:44.617417 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1304/2603] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:41:44 2022] DELETE /identity/v3/users/41e18bc015b545acaa92588c84622a5a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:41:44.802035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:41:44.803681 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:41:44.819349 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2221-3b3e-4ab4-a9cb-3e3634f148dc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:41:44.842397 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:41:44.881122 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:41:44.881847 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:41:44.882127 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:41:44.882400 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] PATH_INFO: `/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:41:44.883560 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0e2b1a36404275a8e53fbbd15b6742)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:41:44.885512 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:41:44.931539 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0e2b1a36404275a8e53fbbd15b6742', 'request_id': 'req-aeed22f9-7002-425c-bc31-45e968ba559f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.937667 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0e2b1a36404275a8e53fbbd15b6742', 'request_id': 'req-aeed22f9-7002-425c-bc31-45e968ba559f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.945942 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0e2b1a36404275a8e53fbbd15b6742', 'request_id': 'req-aeed22f9-7002-425c-bc31-45e968ba559f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.956541 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] The token cache is being invalidate because project ef0e2b1a36404275a8e53fbbd15b6742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:41:44.956829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.957496 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:41:44.959060 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1300/2604] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:41:44 2022] DELETE /identity/v3/projects/ef0e2b1a36404275a8e53fbbd15b6742 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 13:41:54.616771 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=4 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:41:54.953465 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:42:19.088256 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:42:19.090628 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:19.154612 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7703ed-cec2-4a8a-9912-3a57c96b9daa admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:42:19.213709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:42:19.214321 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:42:19.214608 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:42:19.214884 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:42:19.216938 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:19.288026 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:42:19.289478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea01236-b210-4687-b7db-a45c854bd771 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:42:19.300653 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1305/2605] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:42:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:42:20.487930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:42:20.489475 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:20.502144 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed22f9-7002-425c-bc31-45e968ba559f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:42:20.506035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:42:20.506630 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:42:20.506902 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:42:20.507183 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:42:20.508994 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:20.522665 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:42:20.524096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:42:20.535225 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1301/2606] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:42:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 13:42:22.824523 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:42:22.825040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:42:22.825040 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:42:22.844533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] MFA Rules not processed for user `76d8100e29a24533a014b8d073e45ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:42:22.845736 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:22.911696 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1306/2607] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:42:30.530315 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:42:32.904984 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=99 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:42:43.459107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:42:43.460930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:43.475635 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ee094-3794-4ca1-bc4f-953da52340ef service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:42:43.482055 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:42:43.482677 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:42:43.482956 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:42:43.483228 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:42:43.485278 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:42:43.497857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:42:43.499450 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:42:43.510739 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1302/2608] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:42:53.506855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:43:27.179020 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:43:27.180848 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:43:27.194194 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd169678-2984-44c0-bc54-90c602d5ee45 None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:43:27.198655 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:43:27.199266 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:43:27.199532 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:43:27.199799 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:43:27.201849 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:43:27.211631 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:43:27.213103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92421e2f-4df9-4884-a6f4-0b0325fb6243 service neutron] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:43:27.221221 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1307/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:43:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:43:37.217893 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:44:32.321152 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:44:32.322673 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:32.383974 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cbba4-b425-423d-bcda-3a17c3bb69ff service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:44:32.391113 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ecc3d37e6214164a01eb96629509215', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:44:32.391934 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:32.392352 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:32.392741 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:32.395855 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:32.412558 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:44:32.414008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:44:32.423443 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1303/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:44:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:44:42.419166 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=26 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:44:42.523027 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] REQUEST_METHOD: `POST` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:42.523578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:42.523578 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:42.544159 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.auth.core [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] MFA Rules not processed for user `76d8100e29a24533a014b8d073e45ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81370) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 13:44:42.545332 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:42.552546 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1308/2611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 11 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 13:44:42.892171 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:44:42.903521 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106d51d-9c7c-440c-9663-e545cb2c6279 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:44:42.908107 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:44:42.908738 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:42.909018 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:42.909281 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:42.911389 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:42.922173 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:44:42.923611 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:44:42.932167 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1304/2612] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:44:43.397711 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:44:43.399129 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:43.472223 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d3440-cc0b-415f-acb7-a558eed893ab None None] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:44:43.477448 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:44:43.478135 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:43.478426 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:43.478678 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:43.480486 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:43.491594 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:44:43.493189 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:44:43.502885 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1309/2613] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:44:46.142331 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:44:46.144138 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:46.165729 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b4cef-98c8-4c13-9b99-f48d6be7c9cd service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:44:46.173548 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:44:46.174184 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:46.174435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:46.174710 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:46.176829 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:46.190518 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:44:46.192156 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:44:46.202992 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1305/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:44:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:44:53.499015 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=35 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:44:56.198386 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=12 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:44:58.043422 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:44:58.059161 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da482f6-103e-4c1b-88b9-ba3959f62cfb service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:44:58.065152 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edc1cd61b66547b2a388b1ead24a47c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:44:58.065775 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:44:58.066057 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:44:58.066313 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:44:58.068192 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:44:58.081046 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:44:58.082622 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:44:58.093408 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1310/2615] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:44:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:45:01.416774 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:45:01.427185 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bdf49-b563-4ab5-a3b8-d12705fafb21 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:45:01.431711 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:45:01.432347 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:45:01.432618 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:45:01.432881 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:45:01.435002 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:45:01.501350 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:45:01.502792 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:45:01.512910 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1306/2616] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:45:08.088709 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=6 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:45:11.508129 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=30 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:45:26.474398 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:45:26.476780 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:45:26.567807 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a1c94-5300-42ac-95a7-5c31083dfb01 service nova] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:45:26.573789 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:45:26.574414 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:45:26.574690 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:45:26.574966 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:45:26.576958 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:45:26.589603 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:45:26.591184 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:45:26.601745 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1311/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:45:36.597317 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=37 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:47:19.074388 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:19.101930 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdbdc9-6b47-4a9a-8272-b4dd21bfaa2d service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:19.105713 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07bb14ad0a147fda168a9165a74b7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:19.106316 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:19.106591 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:19.106857 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:19.108621 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:47:19.197283 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:19.198740 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:19.219187 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1307/2618] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:47:22.360818 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:22.372431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec52ce-2017-4288-9e80-4cf450606b8b service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:22.378093 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:22.378824 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] REQUEST_METHOD: `GET` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:22.379103 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:22.379368 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:22.381174 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:47:22.395291 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:22.396929 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:22.407346 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1312/2619] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:47:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:47:25.364790 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:25.366422 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:47:25.379725 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ddf72-9d04-4ea8-92e8-b24d1bbf31d4 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:25.384343 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:25.384981 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:25.385256 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:25.385525 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] PATH_INFO: `/v3/users/76d8100e29a24533a014b8d073e45ca5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:25.387008 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d8100e29a24533a014b8d073e45ca5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:25.388966 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:25.438561 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d8100e29a24533a014b8d073e45ca5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.445914 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d8100e29a24533a014b8d073e45ca5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.458282 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d8100e29a24533a014b8d073e45ca5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.459111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d8100e29a24533a014b8d073e45ca5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.467816 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d8100e29a24533a014b8d073e45ca5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.475435 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1308/2620] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:47:25 2022] DELETE /identity/v3/users/76d8100e29a24533a014b8d073e45ca5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:47:25.686044 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:25.697493 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41921-070b-41c5-888b-7c0cba9d4d04 service placement] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:25.763538 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:25.764225 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:25.764506 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:25.764783 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] PATH_INFO: `/v3/projects/340df5a9e41940d7b010168feb108bd3` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:25.765899 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340df5a9e41940d7b010168feb108bd3)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:25.767814 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:25.815901 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340df5a9e41940d7b010168feb108bd3', 'request_id': 'req-247a36e2-7bb1-4a68-a11e-63107b8847d7'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.822142 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340df5a9e41940d7b010168feb108bd3', 'request_id': 'req-247a36e2-7bb1-4a68-a11e-63107b8847d7'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.830431 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340df5a9e41940d7b010168feb108bd3', 'request_id': 'req-247a36e2-7bb1-4a68-a11e-63107b8847d7'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.841576 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] The token cache is being invalidate because project 340df5a9e41940d7b010168feb108bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:47:25.841858 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.842499 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.844030 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1313/2621] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:47:25 2022] DELETE /identity/v3/projects/340df5a9e41940d7b010168feb108bd3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:47:25.848387 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:25.849871 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:47:25.863096 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b3b5-5546-4025-b25f-35b2afc641f9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:25.875111 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:47:25.929408 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:25.930058 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:25.930333 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:25.930599 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] PATH_INFO: `/v3/users/369f348c7ca44e6baa56bc3e9f7adbb4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:25.931752 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=369f348c7ca44e6baa56bc3e9f7adbb4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:25.933870 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:25.984011 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369f348c7ca44e6baa56bc3e9f7adbb4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:25.990702 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369f348c7ca44e6baa56bc3e9f7adbb4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.003556 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369f348c7ca44e6baa56bc3e9f7adbb4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.004326 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369f348c7ca44e6baa56bc3e9f7adbb4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.014626 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369f348c7ca44e6baa56bc3e9f7adbb4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.020502 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1309/2622] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 11 13:47:25 2022] DELETE /identity/v3/users/369f348c7ca44e6baa56bc3e9f7adbb4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:47:26.199779 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Authenticating user token {{(pid=81370) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:47:26.201535 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.fernet_utils [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81370) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:47:26.215605 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a36e2-7bb1-4a68-a11e-63107b8847d7 admin admin] Validating token access rules against request {{(pid=81370) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:47:26.273213 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c24104e3c5bf48be865c100865a4abd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81928fdc604942f18e8f5f0944263250', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81370) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:47:26.273853 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:47:26.274127 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:47:26.274395 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] PATH_INFO: `/v3/projects/887cdd2afaaa48fe930f5fd87388fc18` {{(pid=81370) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:47:26.275517 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887cdd2afaaa48fe930f5fd87388fc18)` {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:47:26.277456 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] RBAC: Authorization granted {{(pid=81370) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:47:26.298061 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:47:26.325533 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887cdd2afaaa48fe930f5fd87388fc18', 'request_id': 'req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.332478 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887cdd2afaaa48fe930f5fd87388fc18', 'request_id': 'req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.341480 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887cdd2afaaa48fe930f5fd87388fc18', 'request_id': 'req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1'} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.352021 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] The token cache is being invalidate because project 887cdd2afaaa48fe930f5fd87388fc18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81370) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 13:47:26.352334 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.352954 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG keystone.notifications [None req-e3cfb9a3-32d8-4d8c-8824-22c55d65d6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81370) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 13:47:26.354472 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: [pid: 81370|app: 0|req: 1314/2623] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 11 13:47:26 2022] DELETE /identity/v3/projects/887cdd2afaaa48fe930f5fd87388fc18 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 13:47:36.017378 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:47:36.349169 dv-cinder-850618-5-iscsi devstack@keystone.service[81370]: DEBUG dbcounter [-] [81370] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 13:50:33.466990 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 13:50:33.468662 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:50:33.548367 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d650e-acdb-46e6-a3b1-339f2e5b46d4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 13:50:33.611035 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eabb880b598b47c0b7d7f969d4eebffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04522cf99e1243c7a8903071b9803075', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 13:50:33.611663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 13:50:33.611946 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 13:50:33.612269 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 13:50:33.614099 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 13:50:33.625364 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 13:50:33.627032 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300bbf-671a-4896-b913-5169e6a37367 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 13:50:33.637419 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1310/2624] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 11 13:50:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 13:50:43.632663 dv-cinder-850618-5-iscsi devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=58 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}